vm-test-run-nixbot-gitlab
checks.aarch64-linux.nixbot-gitlab
· build #368
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 3600 seconds6run the VM test script7start all VMs8gitlab: starting vm9gitlab # Disk image does not exist, creating the virtualisation disk image...10gitlab: QEMU running (pid 46)11gitlab # Formatting '/build/vm-state-gitlab/tmp.FtJwmD62wB', fmt=raw size=107374182412gitlab # mke2fs 1.47.4 (6-Mar-2025)13gitlab # Discarding device blocks: 0/262144 done14gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes15gitlab # Filesystem UUID: bccaf804-7215-4b9a-b488-51c2452cc90016gitlab # Superblock backups stored on blocks:17(finished: start all VMs, in 69.59 seconds)18gitlab # 32768, 98304, 163840, 22937619gitlab: waiting for unit setup-gitlab.service20gitlab # 21gitlab: waiting for the VM to finish booting22gitlab # Allocating group tables: 0/8 done23gitlab # Writing inode tables: 0/8 done24gitlab # Creating journal (8192 blocks): done25gitlab # Writing superblocks and filesystem accounting information: 0/8 done26gitlab # 27gitlab # Virtualisation disk image created.28gitlab # Creating Nix store image...29gitlab # Created Nix store image.30gitlab # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]31gitlab # [ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Thu Aug 6 17:27:09 UTC 202632gitlab # [ 0.000000] KASLR enabled33gitlab # [ 0.000000] random: crng init done34gitlab # [ 0.000000] Machine model: linux,dummy-virt35gitlab # [ 0.000000] efi: UEFI not found.36gitlab # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT37gitlab # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x00000001bfffffff]38gitlab # [ 0.000000] NODE_DATA(0) allocated [mem 0x1bfdeb740-0x1bfdeeebf]39gitlab # [ 0.000000] Zone ranges:40gitlab # [ 0.000000] DMA [mem 0x0000000040000000-0x00000000ffffffff]41gitlab # [ 0.000000] DMA32 empty42gitlab # [ 0.000000] Normal [mem 0x0000000100000000-0x00000001bfffffff]43gitlab # [ 0.000000] Device empty44gitlab # [ 0.000000] Movable zone start for each node45gitlab # [ 0.000000] Early memory node ranges46gitlab # [ 0.000000] node 0: [mem 0x0000000040000000-0x00000001bfffffff]47gitlab # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000001bfffffff]48gitlab # [ 0.000000] cma: Reserved 32 MiB at 0x00000000fe00000049gitlab # [ 0.000000] psci: probing for conduit method from DT.50gitlab # [ 0.000000] psci: PSCIv1.3 detected in firmware.51gitlab # [ 0.000000] psci: Using standard PSCI v0.2 function IDs52gitlab # [ 0.000000] psci: Trusted OS migration not required53gitlab # [ 0.000000] psci: SMC Calling Convention v1.154gitlab # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)55gitlab # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u31129656gitlab # [ 0.000000] Detected PIPT I-cache on CPU057gitlab # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)58gitlab # [ 0.000000] CPU features: detected: GICv3 CPU interface59gitlab # [ 0.000000] CPU features: detected: Spectre-v460gitlab # [ 0.000000] CPU features: detected: Spectre-BHB61gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_3862gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_2363gitlab # [ 0.000000] alternatives: applying boot alternatives64gitlab # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/35qpkln0cfla9fyi3jx6h9il8yw6cr88-nixos-system-gitlab-test/init regInfo=/nix/store/m3gdmpgf53k7p2vvlm34mbffmmcbbx5q-closure-info/registration console=ttyAMA0,115200n8 console=tty065gitlab # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/m3gdmpgf53k7p2vvlm34mbffmmcbbx5q-closure-info/registration", will be passed to user space.66gitlab # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes67gitlab # [ 0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)68gitlab # [ 0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)69gitlab # [ 0.000000] software IO TLB: area num 4.70gitlab # [ 0.000000] software IO TLB: mapped [mem 0x00000000fa000000-0x00000000fe000000] (64MB)71gitlab # [ 0.000000] Fallback order for Node 0: 072gitlab # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 157286473gitlab # [ 0.000000] Policy zone: Normal74gitlab # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off75gitlab # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=176gitlab # [ 0.000000] allocated 12582912 bytes of page_ext77gitlab # [ 0.000000] ftrace: allocating 74867 entries in 294 pages78gitlab # [ 0.000000] ftrace: allocated 294 pages with 4 groups79gitlab # [ 0.000000] rcu: Hierarchical RCU implementation.80gitlab # [ 0.000000] rcu: RCU event tracing is enabled.81gitlab # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.82gitlab # [ 0.000000] Trampoline variant of Tasks RCU enabled.83gitlab # [ 0.000000] Rude variant of Tasks RCU enabled.84gitlab # [ 0.000000] Tracing variant of Tasks RCU enabled.85gitlab # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.86gitlab # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=487gitlab # [ 0.000000] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.88gitlab # [ 0.000000] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.89gitlab # [ 0.000000] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.90gitlab # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 091gitlab # [ 0.000000] GICv3: 256 SPIs implemented92gitlab # [ 0.000000] GICv3: 0 Extended SPIs implemented93gitlab # [ 0.000000] Root IRQ handler: gic_handle_irq94gitlab # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI95gitlab # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=096gitlab # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a000097gitlab # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]98gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @100f10000 (indirect, esz 8, psz 64K, shr 1)99gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @100f20000 (flat, esz 8, psz 64K, shr 1)100gitlab # [ 0.000000] GICv3: using LPI property table @0x0000000100f30000101gitlab # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000100f40000102gitlab # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.103gitlab # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns104gitlab # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).105gitlab # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns106gitlab # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns107gitlab # [ 0.000028] arm-pv: using stolen time PV108gitlab # [ 0.000382] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)109gitlab # [ 0.000566] Console: colour dummy device 80x25110gitlab # [ 0.000574] printk: legacy console [tty0] enabled111gitlab # [ 0.000753] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)112gitlab # [ 0.000759] pid_max: default: 32768 minimum: 301113gitlab # [ 0.000836] LSM: initializing lsm=capability,landlock,yama,bpf,ima114gitlab # [ 0.000967] landlock: Up and running.115gitlab # [ 0.000970] Yama: becoming mindful.116gitlab # [ 0.001393] LSM support for eBPF active117gitlab # [ 0.001604] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)118gitlab # [ 0.001717] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)119gitlab # [ 0.003229] cacheinfo: Unable to detect cache hierarchy for CPU 0120gitlab # [ 0.003902] rcu: Hierarchical SRCU implementation.121gitlab # [ 0.003906] rcu: Max phase no-delay instances is 1000.122gitlab # [ 0.004083] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level123gitlab # [ 0.005063] fsl-mc MSI: its@8080000 domain created124gitlab # [ 0.005156] EFI services will not be available.125gitlab # [ 0.005332] smp: Bringing up secondary CPUs ...126gitlab # [ 0.005979] Detected PIPT I-cache on CPU1127gitlab # [ 0.006081] GICv3: CPU1: found redistributor 1 region 0:0x00000000080c0000128gitlab # [ 0.006212] GICv3: CPU1: using allocated LPI pending table @0x0000000100f50000129gitlab # [ 0.006339] CPU1: Booted secondary processor 0x0000000001 [0xc00fac40]130gitlab # [ 0.007453] Detected PIPT I-cache on CPU2131gitlab # [ 0.007562] GICv3: CPU2: found redistributor 2 region 0:0x00000000080e0000132gitlab # [ 0.007695] GICv3: CPU2: using allocated LPI pending table @0x0000000100f60000133gitlab # [ 0.007829] CPU2: Booted secondary processor 0x0000000002 [0xc00fac40]134gitlab # [ 0.008935] Detected PIPT I-cache on CPU3135gitlab # [ 0.009053] GICv3: CPU3: found redistributor 3 region 0:0x0000000008100000136gitlab # [ 0.009186] GICv3: CPU3: using allocated LPI pending table @0x0000000100f70000137gitlab # [ 0.009318] CPU3: Booted secondary processor 0x0000000003 [0xc00fac40]138gitlab # [ 0.009822] smp: Brought up 1 node, 4 CPUs139gitlab # [ 0.009847] SMP: Total of 4 processors activated.140gitlab # [ 0.009850] CPU: All CPU(s) started at EL1141gitlab # [ 0.009858] CPU features: detected: Branch Target Identification142gitlab # [ 0.009862] CPU features: detected: ARMv8.4 Translation Table Level143gitlab # [ 0.009864] CPU features: detected: Instruction cache invalidation not required for I/D coherence144gitlab # [ 0.009868] CPU features: detected: Data cache clean to the PoU not required for I/D coherence145gitlab # [ 0.009871] CPU features: detected: Common not Private translations146gitlab # [ 0.009874] CPU features: detected: CRC32 instructions147gitlab # [ 0.009876] CPU features: detected: Data cache clean to Point of Deep Persistence148gitlab # [ 0.009880] CPU features: detected: Data cache clean to Point of Persistence149gitlab # [ 0.009883] CPU features: detected: Data independent timing control (DIT)150gitlab # [ 0.009885] CPU features: detected: E0PD151gitlab # [ 0.009888] CPU features: detected: Enhanced Counter Virtualization152gitlab # [ 0.009890] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)153gitlab # [ 0.009893] CPU features: detected: Enhanced Virtualization Traps154gitlab # [ 0.009896] CPU features: detected: Fine Grained Traps155gitlab # [ 0.009899] CPU features: detected: Generic authentication (architected QARMA5 algorithm)156gitlab # [ 0.009903] CPU features: detected: RCpc load-acquire (LDAPR)157gitlab # [ 0.009905] CPU features: detected: LSE atomic instructions158gitlab # [ 0.009908] CPU features: detected: Privileged Access Never159gitlab # [ 0.009910] CPU features: detected: PMUv3160gitlab # [ 0.009912] CPU features: detected: RAS Extension Support161gitlab # [ 0.009914] CPU features: detected: RASv1p1 Extension Support162gitlab # [ 0.009917] CPU features: detected: Random Number Generator163gitlab # [ 0.009919] CPU features: detected: Speculation barrier (SB)164gitlab # [ 0.009921] CPU features: detected: Stage-2 Force Write-Back165gitlab # [ 0.009924] CPU features: detected: TLB range maintenance instructions166gitlab # [ 0.009927] CPU features: detected: Speculative Store Bypassing Safe (SSBS)167gitlab # [ 0.010068] alternatives: applying system-wide alternatives168gitlab # [ 0.012937] CPU features: detected: BBM Level 2 without TLB conflict abort169gitlab # [ 0.013153] Memory: 5963828K/6291456K available (24448K kernel code, 7090K rwdata, 26340K rodata, 4736K init, 1106K bss, 272908K reserved, 32768K cma-reserved)170gitlab # [ 0.014575] devtmpfs: initialized171gitlab # [ 0.018696] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)172gitlab # [ 0.018763] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).173gitlab # [ 0.018925] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL174gitlab # [ 0.018930] 0 pages in range for non-PLT usage175gitlab # [ 0.018931] 508288 pages in range for PLT usage176gitlab # [ 0.019057] pinctrl core: initialized pinctrl subsystem177gitlab # [ 0.019806] DMI not present or invalid.178gitlab # [ 0.023070] NET: Registered PF_NETLINK/PF_ROUTE protocol family179gitlab # [ 0.025851] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations180gitlab # [ 0.026120] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations181gitlab # [ 0.027984] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations182gitlab # [ 0.028004] audit: initializing netlink subsys (disabled)183gitlab # [ 0.028315] audit: type=2000 audit(0.024:1): state=initialized audit_enabled=0 res=1184gitlab # [ 0.028772] thermal_sys: Registered thermal governor 'fair_share'185gitlab # [ 0.028775] thermal_sys: Registered thermal governor 'bang_bang'186gitlab # [ 0.028779] thermal_sys: Registered thermal governor 'step_wise'187gitlab # [ 0.028782] thermal_sys: Registered thermal governor 'user_space'188gitlab # [ 0.028784] thermal_sys: Registered thermal governor 'power_allocator'189gitlab # [ 0.028866] cpuidle: using governor ladder190gitlab # [ 0.028928] cpuidle: using governor menu191gitlab # [ 0.029129] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.192gitlab # [ 0.029335] ASID allocator initialised with 65536 entries193gitlab # [ 0.030428] Serial: AMBA PL011 UART driver194gitlab # [ 0.035551] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1195gitlab # [ 0.035831] printk: console [ttyAMA0] enabled196gitlab # [ 0.047980] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages197gitlab # [ 0.047998] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page198gitlab # [ 0.048008] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages199gitlab # [ 0.048016] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page200gitlab # [ 0.048024] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages201gitlab # [ 0.048032] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page202gitlab # [ 0.048041] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages203gitlab # [ 0.048048] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page204gitlab # [ 0.059692] fbcon: Taking over console205gitlab # [ 0.059703] ACPI: Interpreter disabled.206gitlab # [ 0.060717] iommu: Default domain type: Translated207gitlab # [ 0.060734] iommu: DMA domain TLB invalidation policy: strict mode208gitlab # [ 0.070602] SCSI subsystem initialized209gitlab # [ 0.070981] usbcore: registered new interface driver usbfs210gitlab # [ 0.071003] usbcore: registered new interface driver hub211gitlab # [ 0.071017] usbcore: registered new device driver usb212gitlab # [ 0.071315] pps_core: LinuxPPS API ver. 1 registered213gitlab # [ 0.071318] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>214gitlab # [ 0.071327] PTP clock support registered215gitlab # [ 0.071363] EDAC MC: Ver: 3.0.0216gitlab # [ 0.071631] scmi_core: SCMI protocol bus registered217gitlab # [ 0.072194] FPGA manager framework218gitlab # [ 0.072895] vgaarb: loaded219gitlab # [ 0.073766] clocksource: Switched to clocksource arch_sys_counter220gitlab # [ 0.074213] VFS: Disk quotas dquot_6.6.0221gitlab # [ 0.074236] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)222gitlab # [ 0.077652] netfs: FS-Cache loaded223gitlab # [ 0.077748] pnp: PnP ACPI: disabled224gitlab # [ 0.085180] NET: Registered PF_INET protocol family225gitlab # [ 0.086208] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)226gitlab # [ 0.115477] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)227gitlab # [ 0.115523] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)228gitlab # [ 0.115565] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)229gitlab # [ 0.115819] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)230gitlab # [ 0.116369] TCP: Hash tables configured (established 65536 bind 65536)231gitlab # [ 0.116454] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)232gitlab # [ 0.116502] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)233gitlab # [ 0.116612] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)234gitlab # [ 0.116761] NET: Registered PF_UNIX/PF_LOCAL protocol family235gitlab # [ 0.116809] NET: Registered PF_XDP protocol family236gitlab # [ 0.116822] PCI: CLS 0 bytes, default 64237gitlab # [ 0.117084] Trying to unpack rootfs image as initramfs...238gitlab # [ 0.129921] kvm [1]: HYP mode not available239gitlab # [ 0.216522] Initialise system trusted keyrings240gitlab # [ 0.216963] workingset: timestamp_bits=42 max_order=21 bucket_order=0241gitlab # [ 0.218152] squashfs: version 4.0 (2009/01/31) Phillip Lougher242gitlab # [ 0.218225] 9p: Installing v9fs 9p2000 file system support243gitlab # [ 0.235028] Key type asymmetric registered244gitlab # [ 0.235034] Asymmetric key parser 'x509' registered245gitlab # [ 0.235138] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)246gitlab # [ 0.235400] io scheduler mq-deadline registered247gitlab # [ 0.235407] io scheduler kyber registered248gitlab # [ 0.243567] pl061_gpio 9030000.pl061: PL061 GPIO chip registered249gitlab # [ 0.246236] ledtrig-cpu: registered to indicate activity on CPUs250gitlab # [ 0.246769] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:251gitlab # [ 0.246786] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000252gitlab # [ 0.246808] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000253gitlab # [ 0.246815] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000254gitlab # [ 0.246846] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits255gitlab # [ 0.246881] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]256gitlab # [ 0.246969] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00257gitlab # [ 0.246979] pci_bus 0000:00: root bus resource [bus 00-ff]258gitlab # [ 0.246983] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]259gitlab # [ 0.246988] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]260gitlab # [ 0.246992] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]261gitlab # [ 0.247138] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint262gitlab # [ 0.247988] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint263gitlab # [ 0.248377] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]264gitlab # [ 0.248400] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]265gitlab # [ 0.248458] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]266gitlab # [ 0.248481] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]267gitlab # [ 0.249424] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint268gitlab # [ 0.249854] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]269gitlab # [ 0.249871] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]270gitlab # [ 0.249912] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]271gitlab # [ 0.250770] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint272gitlab # [ 0.251062] pci 0000:00:03.0: BAR 0 [io 0x0000-0x001f]273gitlab # [ 0.251078] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]274gitlab # [ 0.251122] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]275gitlab # [ 0.252073] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint276gitlab # [ 0.252472] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]277gitlab # [ 0.252494] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]278gitlab # [ 0.252553] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]279gitlab # [ 0.253457] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint280gitlab # [ 0.253817] pci 0000:00:05.0: BAR 0 [io 0x0000-0x007f]281gitlab # [ 0.253839] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]282gitlab # [ 0.253881] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]283gitlab # [ 0.254637] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint284gitlab # [ 0.255021] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]285gitlab # [ 0.255038] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]286gitlab # [ 0.255085] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]287gitlab # [ 0.255889] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint288gitlab # [ 0.256287] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]289gitlab # [ 0.256309] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]290gitlab # [ 0.256367] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]291gitlab # [ 0.256389] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]292gitlab # [ 0.257261] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint293gitlab # [ 0.257675] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]294gitlab # [ 0.257727] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]295gitlab # [ 0.258642] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint296gitlab # [ 0.258984] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]297gitlab # [ 0.259042] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]298gitlab # [ 0.259774] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint299gitlab # [ 0.260151] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]300gitlab # [ 0.260551] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint301gitlab # [ 0.261036] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]302gitlab # [ 0.261057] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]303gitlab # [ 0.261095] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]304gitlab # [ 0.261981] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint305gitlab # [ 0.262381] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]306gitlab # [ 0.262404] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]307gitlab # [ 0.262461] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]308gitlab # [ 0.263560] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned309gitlab # [ 0.263569] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned310gitlab # [ 0.263574] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned311gitlab # [ 0.263644] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned312gitlab # [ 0.263725] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned313gitlab # [ 0.263792] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned314gitlab # [ 0.263857] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned315gitlab # [ 0.263937] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned316gitlab # [ 0.264003] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned317gitlab # [ 0.264069] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned318gitlab # [ 0.264136] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned319gitlab # [ 0.264202] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned320gitlab # [ 0.264283] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned321gitlab # [ 0.264369] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned322gitlab # [ 0.264404] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned323gitlab # [ 0.264438] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned324gitlab # [ 0.264463] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned325gitlab # [ 0.264489] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned326gitlab # [ 0.264514] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned327gitlab # [ 0.264538] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned328gitlab # [ 0.264563] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned329gitlab # [ 0.264598] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned330gitlab # [ 0.264625] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned331gitlab # [ 0.264650] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned332gitlab # [ 0.264676] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned333gitlab # [ 0.264700] pci 0000:00:05.0: BAR 0 [io 0x1000-0x107f]: assigned334gitlab # [ 0.264724] pci 0000:00:06.0: BAR 0 [io 0x1080-0x10ff]: assigned335gitlab # [ 0.264750] pci 0000:00:0b.0: BAR 0 [io 0x1100-0x113f]: assigned336gitlab # [ 0.264774] pci 0000:00:01.0: BAR 0 [io 0x1140-0x115f]: assigned337gitlab # [ 0.264799] pci 0000:00:02.0: BAR 0 [io 0x1160-0x117f]: assigned338gitlab # [ 0.264824] pci 0000:00:03.0: BAR 0 [io 0x1180-0x119f]: assigned339gitlab # [ 0.264849] pci 0000:00:04.0: BAR 0 [io 0x11a0-0x11bf]: assigned340gitlab # [ 0.264875] pci 0000:00:07.0: BAR 0 [io 0x11c0-0x11df]: assigned341gitlab # [ 0.264900] pci 0000:00:0c.0: BAR 0 [io 0x11e0-0x11ff]: assigned342gitlab # [ 0.264931] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]343gitlab # [ 0.264938] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]344gitlab # [ 0.264942] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]345gitlab # [ 0.267239] pci 0000:00:0a.0: enabling device (0000 -> 0002)346gitlab # [ 0.283214] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)347gitlab # [ 0.286537] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)348gitlab # [ 0.289342] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)349gitlab # [ 0.292291] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)350gitlab # [ 0.298457] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)351gitlab # [ 0.301117] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)352gitlab # [ 0.303884] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)353gitlab # [ 0.306721] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)354gitlab # [ 0.313103] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)355gitlab # [ 0.315708] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)356gitlab # [ 0.319240] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)357gitlab # [ 0.327208] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled358gitlab # [ 0.329431] msm_serial: driver initialized359gitlab # [ 0.329661] SuperH (H)SCI(F) driver initialized360gitlab # [ 0.329725] STM32 USART driver initialized361gitlab # [ 0.355616] loop: module loaded362gitlab # [ 0.355914] virtio_blk virtio4: 4/0/0 default/read/poll queues363gitlab # [ 0.358332] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)364gitlab # [ 0.360609] virtio_blk virtio5: 4/0/0 default/read/poll queues365gitlab # [ 0.362847] virtio_blk virtio5: [vdb] 12285136 512-byte logical blocks (6.29 GB/5.86 GiB)366gitlab # [ 0.365198] megasas: 07.734.00.00-rc1367gitlab # [ 0.366084] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]368gitlab # [ 0.368927] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000369gitlab # [ 0.368951] Intel/Sharp Extended Query Table at 0x0031370gitlab # [ 0.371102] Using buffer write method371gitlab # [ 0.371145] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]372gitlab # [ 0.373472] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000373gitlab # [ 0.373497] Intel/Sharp Extended Query Table at 0x0031374gitlab # [ 0.375737] Using buffer write method375gitlab # [ 0.375757] Concatenating MTD devices:376gitlab # [ 0.375760] (0): "0.flash"377gitlab # [ 0.375763] (1): "0.flash"378gitlab # [ 0.375765] into device "0.flash"379gitlab # [ 0.556275] Freeing initrd memory: 27184K380gitlab # [ 0.576057] tun: Universal TUN/TAP device driver, 1.6381gitlab # [ 0.585181] thunder_xcv, ver 1.0382gitlab # [ 0.585281] thunder_bgx, ver 1.0383gitlab # [ 0.585358] nicpf, ver 1.0384gitlab # [ 0.586972] e1000: Intel(R) PRO/1000 Network Driver385gitlab # [ 0.586983] e1000: Copyright (c) 1999-2006 Intel Corporation.386gitlab # [ 0.587072] e1000e: Intel(R) PRO/1000 Network Driver387gitlab # [ 0.587085] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.388gitlab # [ 0.587151] igb: Intel(R) Gigabit Ethernet Network Driver389gitlab # [ 0.587159] igb: Copyright (c) 2007-2014 Intel Corporation.390gitlab # [ 0.587260] igbvf: Intel(R) Gigabit Virtual Function Network Driver391gitlab # [ 0.587269] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.392gitlab # [ 0.587636] sky2: driver version 1.30393gitlab # [ 0.592089] usbcore: registered new interface driver usb-storage394gitlab # [ 0.592381] usbcore: registered new interface driver usbserial_generic395gitlab # [ 0.592418] usbserial: USB Serial support registered for generic396gitlab # [ 0.592592] ehci-pci 0000:00:0a.0: EHCI Host Controller397gitlab # [ 0.592652] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 1398gitlab # [ 0.593264] ehci-pci 0000:00:0a.0: irq 15, io mem 0x10089000399gitlab # [ 0.594098] hv_vmbus: registering driver hyperv_keyboard400gitlab # [ 0.596548] rtc-pl031 9010000.pl031: registered as rtc0401gitlab # [ 0.596600] rtc-pl031 9010000.pl031: setting system clock to 2026-09-13T10:10:42 UTC (1789294242)402gitlab # [ 0.597430] i2c_dev: i2c /dev entries driver403gitlab # [ 0.605099] sdhci: Secure Digital Host Controller Interface driver404gitlab # [ 0.605112] sdhci: Copyright(c) Pierre Ossman405gitlab # [ 0.605851] Synopsys Designware Multimedia Card Interface Driver406gitlab # [ 0.605948] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.00407gitlab # [ 0.606876] hub 1-0:1.0: USB hub found408gitlab # [ 0.606916] hub 1-0:1.0: 6 ports detected409gitlab # [ 0.606944] sdhci-pltfm: SDHCI platform and OF driver helper410gitlab # [ 0.611120] hid: raw HID events driver (C) Jiri Kosina411gitlab # [ 0.611876] usbcore: registered new interface driver usbhid412gitlab # [ 0.611885] usbhid: USB HID core driver413gitlab # [ 0.628920] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available414gitlab # [ 0.633900] drop_monitor: Initializing network drop monitor service415gitlab # [ 0.634278] NET: Registered PF_INET6 protocol family416gitlab # [ 0.636433] Segment Routing with IPv6417gitlab # [ 0.636471] In-situ OAM (IOAM) with IPv6418gitlab # [ 0.636540] NET: Registered PF_PACKET protocol family419gitlab # [ 0.636942] 9pnet: Installing 9P2000 support420gitlab # [ 0.641484] Key type dns_resolver registered421gitlab # [ 0.660724] registered taskstats version 1422gitlab # [ 0.661462] Loading compiled-in X.509 certificates423gitlab # [ 0.688750] Demotion targets for Node 0: null424gitlab # [ 0.689015] Key type .fscrypt registered425gitlab # [ 0.689024] Key type fscrypt-provisioning registered426gitlab # [ 0.689237] ima: No TPM chip found, activating TPM-bypass!427gitlab # [ 0.689264] ima: Allocated hash algorithm: sha1428gitlab # [ 0.689309] ima: No architecture policies found429gitlab # [ 0.690717] input: gpio-keys as /devices/platform/gpio-keys/input/input0430gitlab # [ 0.723239] clk: Disabling unused clocks431gitlab # [ 0.723272] PM: genpd: Disabling unused power domains432gitlab # [ 0.729115] Freeing unused kernel memory: 4736K433gitlab # [ 0.729340] Run /init as init process434gitlab # [ 0.761384] systemd[1]: Successfully made /usr/ read-only.435gitlab # [ 0.857854] usb 1-1: new high-speed USB device number 2 using ehci-pci436gitlab # [ 1.014822] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1437gitlab # [ 1.096389] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)438gitlab # [ 1.096457] systemd[1]: Detected virtualization qemu.439gitlab # [ 1.096620] systemd[1]: Detected architecture arm64.440gitlab # [ 1.096638] systemd[1]: Running in initrd.441gitlab # [ 1.097494] systemd[1]: Initializing machine ID from random generator.442gitlab # [ 1.097937] systemd[1]: Hostname set to <gitlab>.443gitlab # [ 1.109126] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input0444gitlab # [ 1.225863] usb 1-2: new high-speed USB device number 3 using ehci-pci445gitlab # [ 1.335376] systemd[1]: bpf-restrict-fs: LSM BPF program attached446gitlab # [ 1.379879] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2447gitlab # [ 1.380334] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input0448gitlab # [ 1.414256] systemd[1]: Queued start job for default target Initrd Default Target.449gitlab # [ 1.432835] systemd[1]: Created slice Slice /system/modprobe.450gitlab # [ 1.433096] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.451gitlab # [ 1.433133] systemd[1]: Expecting device /dev/disk/by-label/nix-store...452gitlab # [ 1.433146] systemd[1]: Expecting device /dev/disk/by-label/nixos...453gitlab # [ 1.433173] systemd[1]: Reached target Path Units.454gitlab # [ 1.433190] systemd[1]: Reached target Slice Units.455gitlab # [ 1.433209] systemd[1]: Reached target Swaps.456gitlab # [ 1.433225] systemd[1]: Reached target Timer Units.457gitlab # [ 1.433403] systemd[1]: Listening on D-Bus System Message Bus Socket.458gitlab # [ 1.433577] systemd[1]: Listening on Journal Socket (/dev/log).459gitlab # [ 1.433882] systemd[1]: Listening on Journal Sockets.460gitlab # [ 1.434046] systemd[1]: Listening on udev Control Socket.461gitlab # [ 1.434143] systemd[1]: Listening on udev Kernel Socket.462gitlab # [ 1.434164] systemd[1]: Reached target Socket Units.463gitlab # [ 1.436171] systemd[1]: Starting Create List of Static Device Nodes...464gitlab # [ 1.438166] systemd[1]: Starting Load Kernel Module 9pnet_virtio...465gitlab # [ 1.438243] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs466gitlab # [ 1.440022] systemd[1]: Mounting Kernel Configuration File System...467gitlab # [ 1.467278] systemd[1]: Starting Journal Service...468gitlab # [ 1.471633] systemd[1]: Starting Load Kernel Modules...469gitlab # [ 1.471862] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os470gitlab # [ 1.475261] systemd[1]: Starting Coldplug All udev Devices...471gitlab # [ 1.484284] systemd[1]: Finished Create List of Static Device Nodes.472gitlab # [ 1.486997] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.473gitlab # [ 1.488074] systemd[1]: Finished Load Kernel Module 9pnet_virtio.474gitlab # [ 1.488869] systemd[1]: Mounted Kernel Configuration File System.475gitlab # [ 1.494103] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...476gitlab # [ 1.496617] systemd-journald[95]: Collecting audit messages is disabled.477gitlab # [ 1.518129] [drm] pci: virtio-gpu-pci detected at 0000:00:08.0478gitlab # [ 1.518310] [drm] features: -virgl +edid -resource_blob -host_visible479gitlab # [ 1.518314] [drm] features: -context_init480gitlab # [ 1.519148] [drm] number of scanouts: 1481gitlab # [ 1.519160] [drm] number of cap sets: 0482gitlab # [ 1.526528] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.483gitlab # [ 1.527392] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.484gitlab # [ 1.530150] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev485gitlab # [ 1.531727] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic486gitlab # [ 1.531741] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 0487gitlab # [ 1.538013] systemd[1]: Starting Create Static Device Nodes in /dev...488gitlab # [ 1.539617] Console: switching to colour frame buffer device 160x50489gitlab # [ 1.547348] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device490gitlab # [ 1.556032] systemd[1]: Finished Create Static Device Nodes in /dev.491gitlab # [ 1.556711] systemd[1]: Reached target Preparation for Local File Systems.492gitlab # [ 1.556766] systemd[1]: Reached target Local File Systems.493gitlab # [ 1.563387] systemd[1]: Starting Rule-based Manager for Device Events and Files...494gitlab # [ 1.565629] systemd[1]: Finished Load Kernel Modules.495gitlab # [ 1.574323] systemd[1]: Starting Apply Kernel Variables...496gitlab # [ 1.590392] systemd[1]: Started Journal Service.497gitlab # [ 1.588374] systemd-modules-load[96]: Using 4 probe threads498gitlab # [ 1.589635] systemd-modules-load[96]: Module 'virtio_balloon' is built in499gitlab # [ 1.590725] systemd-modules-load[96]: Module 'virtio_console' is built in500gitlab # [ 1.591786] systemd-modules-load[96]: Module 'virtio_rng' is built in501gitlab # [ 1.596420] systemd-modules-load[96]: Inserted module 'dm_mod'502gitlab # [ 1.597438] systemd-modules-load[96]: Inserted module 'virtio_gpu'503gitlab # [ 1.600267] systemd[1]: Starting Create System Files and Directories...504gitlab # [ 1.603473] systemd[1]: Finished Apply Kernel Variables.505gitlab # [ 1.623980] systemd[1]: Finished Create System Files and Directories.506gitlab # [ 1.632576] systemd-udevd[106]: Using default interface naming scheme 'v261'.507gitlab # [ 1.653155] systemd[1]: Started Rule-based Manager for Device Events and Files.508gitlab # [ 1.680994] systemd[1]: Starting Virtual Console Setup...509gitlab # [ 1.734548] systemd-vconsole-setup[130]: Configuration of first virtual console was skipped, ignoring remaining ones.510gitlab # [ 1.739625] systemd[1]: Finished Virtual Console Setup.511gitlab # [ 2.184717] systemd[1]: Finished Coldplug All udev Devices.512gitlab # [ 2.187527] systemd[1]: Reached target System Initialization.513gitlab # [ 2.190197] systemd[1]: Reached target Basic System.514gitlab # [ 2.360678] (udev-worker)[138]: Network interface NamePolicy= disabled on kernel command line.515gitlab # [ 2.386235] (udev-worker)[148]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.516gitlab # [ 2.389572] (udev-worker)[148]: Network interface NamePolicy= disabled on kernel command line.517gitlab # [ 2.408162] systemd[1]: Found device /dev/disk/by-label/nixos.518gitlab # [ 2.415372] systemd[1]: Found device /dev/disk/by-label/nix-store.519gitlab # [ 2.426590] systemd[1]: Reached target Initrd Root Device.520gitlab # [ 2.429813] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...521gitlab # [ 2.472387] systemd-fsck[153]: nixos: clean, 12/65536 files, 13019/262144 blocks522gitlab # [ 2.476406] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.523gitlab # [ 2.480216] systemd[1]: Mounting /sysroot...524gitlab # [ 2.521405] EXT4-fs (vda): mounted filesystem bccaf804-7215-4b9a-b488-51c2452cc900 r/w with ordered data mode. Quota mode: none.525gitlab # [ 2.519794] systemd[1]: Mounted /sysroot.526gitlab # [ 2.520514] systemd[1]: Reached target Initrd Root File System.527gitlab # [ 2.523278] systemd[1]: Mounting /sysroot/nix/.ro-store...528gitlab # [ 2.527595] systemd[1]: Mounting /sysroot/nix/.rw-store...529gitlab # [ 2.529554] systemd[1]: Mounting /sysroot/run...530gitlab # [ 2.536441] systemd[1]: Mounting /sysroot/tmp/shared...531gitlab # [ 2.556421] systemd[1]: Mounting /sysroot/tmp/xchg...532gitlab # [ 2.562874] systemd[1]: Starting Mountpoints Configured in the Real Root...533gitlab # [ 2.580332] systemd-sysroot-fstab-check[166]: /sysroot should be mounted in the initrd, will request daemon-reload.534gitlab # [ 2.584698] systemd[1]: Mounted /sysroot/nix/.rw-store.535gitlab # [ 2.585666] systemd[1]: Mounted /sysroot/run.536gitlab # [ 2.586466] systemd[1]: Mounted /sysroot/tmp/xchg.537gitlab # [ 2.587779] systemd[1]: Reload requested from client PID 166 ('systemd-sysroot') (unit initrd-parse-etc.service)...538gitlab # [ 2.589734] systemd[1]: Reloading...539gitlab # [ 2.850005] erofs (device vdb): mounted with root inode @ nid 128.540gitlab # [ 2.899986] systemd[1]: Reloading finished in 311 ms.541gitlab # [ 2.965301] systemd-sysroot-fstab-check[166]: Requesting initrd-fs.target/start/replace...542gitlab # [ 2.969105] systemd[1]: Mounted /sysroot/nix/.ro-store.543gitlab # [ 2.970326] systemd[1]: Mounted /sysroot/tmp/shared.544gitlab # [ 2.973486] systemd-sysroot-fstab-check[166]: Requesting swap.target/start/replace...545gitlab # [ 2.976181] systemd[1]: Starting rw-sysroot-nix-store.service...546gitlab # [ 2.977156] systemd[1]: initrd-parse-etc.service: Deactivated successfully.547gitlab # [ 2.978553] systemd[1]: Finished Mountpoints Configured in the Real Root.548gitlab # [ 2.979979] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.549gitlab # [ 3.001020] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.550gitlab # [ 3.002585] systemd[1]: Finished rw-sysroot-nix-store.service.551gitlab # [ 3.003884] (udev-worker)[136]: mtd0ro: Failed to find and pin callout binary "/nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/mtd_probe": No such file or directory552gitlab # [ 3.006853] (udev-worker)[136]: mtd0ro: /nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory553gitlab # [ 3.011229] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.554gitlab # [ 3.012482] systemd[1]: Stopped Virtual Console Setup.555gitlab # [ 3.013232] systemd[1]: Stopping Virtual Console Setup...556gitlab # [ 3.014033] systemd[1]: Starting Virtual Console Setup...557gitlab # [ 3.036801] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.558gitlab # [ 3.037822] systemd[1]: Stopped Virtual Console Setup.559gitlab # [ 3.038552] systemd[1]: Starting Virtual Console Setup...560gitlab # [ 3.060694] systemd-vconsole-setup[199]: Configuration of first virtual console was skipped, ignoring remaining ones.561gitlab # [ 3.062781] systemd[1]: Finished Virtual Console Setup.562gitlab # [ 3.479503] systemd[1]: Mounting /sysroot/nix/store...563gitlab # [ 3.553487] systemd[1]: Mounted /sysroot/nix/store.564gitlab # [ 3.555818] systemd[1]: Reached target Initrd File Systems.565gitlab # [ 3.558340] systemd[1]: Starting Find NixOS closure...566gitlab # [ 3.561270] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...567gitlab # [ 3.615448] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.568gitlab # [ 3.619412] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.569gitlab # [ 3.634852] systemd[1]: Finished Find NixOS closure.570gitlab # [ 3.637288] systemd[1]: Reached target Initrd Default Target.571gitlab # [ 3.639749] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...572gitlab # [ 3.688632] systemd[1]: Stopped target Initrd Default Target.573gitlab # [ 3.691115] systemd[1]: Stopped target Basic System.574gitlab # [ 3.693460] systemd[1]: Stopped target Initrd Root Device.575gitlab # [ 3.695797] systemd[1]: Stopped target Path Units.576gitlab # [ 3.698038] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.577gitlab # [ 3.701283] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.578gitlab # [ 3.704551] systemd[1]: Stopped target Slice Units.579gitlab # [ 3.706659] systemd[1]: Stopped target Socket Units.580gitlab # [ 3.717127] systemd[1]: Stopped target System Initialization.581gitlab # [ 3.720882] systemd[1]: Stopped target Swaps.582gitlab # [ 3.725055] systemd[1]: Stopped target Timer Units.583gitlab # [ 3.727396] systemd[1]: dbus.socket: Deactivated successfully.584gitlab # [ 3.730189] systemd[1]: Closed D-Bus System Message Bus Socket.585gitlab # [ 3.733684] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.586gitlab # [ 3.736347] systemd[1]: Stopped Find NixOS closure.587gitlab # [ 3.738116] systemd[1]: Starting Load Kernel Module 9pnet_virtio...588gitlab # [ 3.740154] systemd[1]: Starting rw-sysroot-nix-store.service...589gitlab # [ 3.741995] systemd[1]: systemd-sysctl.service: Deactivated successfully.590gitlab # [ 3.743970] systemd[1]: Stopped Apply Kernel Variables.591gitlab # [ 3.745530] systemd[1]: systemd-modules-load.service: Deactivated successfully.592gitlab # [ 3.747440] systemd[1]: Stopped Load Kernel Modules.593gitlab # [ 3.749047] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.594gitlab # [ 3.751122] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.595gitlab # [ 3.753230] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.596gitlab # [ 3.755144] systemd[1]: Stopped Create System Files and Directories.597gitlab # [ 3.756970] systemd[1]: Stopped target Local File Systems.598gitlab # [ 3.758541] systemd[1]: Stopped target Preparation for Local File Systems.599gitlab # [ 3.760417] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.600gitlab # [ 3.762325] systemd[1]: Stopped Coldplug All udev Devices.601gitlab # [ 3.763835] systemd[1]: Stopping Rule-based Manager for Device Events and Files...602gitlab # [ 3.765913] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.603gitlab # [ 3.767813] systemd[1]: Stopped Virtual Console Setup.604gitlab # [ 3.769315] systemd[1]: initrd-cleanup.service: Deactivated successfully.605gitlab # [ 3.771055] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.606gitlab # [ 3.772835] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.607gitlab # [ 3.774696] systemd[1]: Finished rw-sysroot-nix-store.service.608gitlab # [ 3.776245] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.609gitlab # [ 3.778139] systemd[1]: Finished Load Kernel Module 9pnet_virtio.610gitlab # [ 3.779731] systemd[1]: systemd-udevd.service: Deactivated successfully.611gitlab # [ 3.781604] systemd[1]: Stopped Rule-based Manager for Device Events and Files.612gitlab # [ 3.783382] systemd[1]: systemd-udevd.service: Consumed 2.316s CPU time over 2.200s wall clock time, 31.6M memory peak.613gitlab # [ 3.786942] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.614gitlab # [ 3.788653] systemd[1]: Closed udev Control Socket.615gitlab # [ 3.789881] systemd[1]: Starting Cleanup udev Database...616gitlab # [ 3.791390] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.617gitlab # [ 3.793319] systemd[1]: Stopped Create Static Device Nodes in /dev.618gitlab # [ 3.794824] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.619gitlab # [ 3.796801] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.620gitlab # [ 3.798502] systemd[1]: kmod-static-nodes.service: Deactivated successfully.621gitlab # [ 3.800160] systemd[1]: Stopped Create List of Static Device Nodes.622gitlab # [ 3.869040] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.623gitlab # [ 3.872291] systemd[1]: Finished Cleanup udev Database.624gitlab # [ 3.874540] systemd[1]: Reached target Switch Root.625gitlab # [ 3.876761] systemd[1]: Starting NixOS Activation...626gitlab # [ 4.058619] initrd-nixos-activation-start[224]: booting system configuration /nix/store/35qpkln0cfla9fyi3jx6h9il8yw6cr88-nixos-system-gitlab-test627gitlab # [ 4.121720] initrd-nixos-activation-start[224]: running activation script...628gitlab # [ 4.442936] initrd-nixos-activation-start[247]: setting up /etc...629gitlab # [ 4.592502] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.630gitlab # [ 4.595878] systemd[1]: Finished NixOS Activation.631gitlab # [ 4.598222] systemd[1]: Starting Switch Root...632gitlab # [ 4.654727] systemd[1]: Switching root.633gitlab # [ 4.762663] systemd-journald[95]: Received SIGTERM from PID 1 (systemd).634gitlab # [ 5.406339] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)635gitlab # [ 5.406571] systemd[1]: Detected virtualization qemu.636gitlab # [ 5.406704] systemd[1]: Detected architecture arm64.637gitlab # [ 5.407079] systemd[1]: Detected first boot.638gitlab # [ 5.409450] systemd[1]: Initializing machine ID from random generator.639gitlab # [ 5.687834] systemd[1]: bpf-restrict-fs: LSM BPF program attached640gitlab # [ 5.854903] systemd[1]: Applying preset policy.641gitlab # [ 6.093327] systemd[1]: Populated /etc with preset unit settings.642gitlab # [ 6.313370] systemd[1]: initrd-switch-root.service: Deactivated successfully.643gitlab # [ 6.313913] systemd[1]: Stopped initrd-switch-root.service.644gitlab # [ 6.316856] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.645gitlab # [ 6.321240] systemd[1]: Created slice Slice /system/getty.646gitlab # [ 6.327477] systemd[1]: Created slice GitLab DevOps Platform Slice.647gitlab # [ 6.331067] systemd[1]: Created slice User and Session Slice.648gitlab # [ 6.331740] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.649gitlab # [ 6.332511] systemd[1]: Started Forward Password Requests to Wall Directory Watch.650gitlab # [ 6.333187] systemd[1]: Expecting device /dev/hvc0...651gitlab # [ 6.338154] systemd[1]: Expecting device /dev/ttyAMA0...652gitlab # [ 6.339710] systemd[1]: Reached target Local Encrypted Volumes.653gitlab # [ 6.341408] systemd[1]: Stopped target initrd-fs.target.654gitlab # [ 6.342888] systemd[1]: Stopped target initrd-root-fs.target.655gitlab # [ 6.344177] systemd[1]: Stopped target initrd-switch-root.target.656gitlab # [ 6.345411] systemd[1]: Reached target Virtual Machines and Containers.657gitlab # [ 6.346697] systemd[1]: Reached target Path Units.658gitlab # [ 6.348028] systemd[1]: Reached target Remote File Systems.659gitlab # [ 6.349816] systemd[1]: Reached target Slice Units.660gitlab # [ 6.351378] systemd[1]: Reached target Swaps.661gitlab # [ 6.357309] systemd[1]: Listening on Query the User Interactively for a Password.662gitlab # [ 6.364088] systemd[1]: Listening on Process Core Dump Socket.663gitlab # [ 6.368817] systemd[1]: Listening on Credential Encryption/Decryption.664gitlab # [ 6.374189] systemd[1]: Listening on Factory Reset Management.665gitlab # [ 6.376023] systemd[1]: Listening on Hostname Service Socket.666gitlab # [ 6.390097] systemd[1]: Starting Journal Log Access Socket...667gitlab # [ 6.392918] systemd[1]: Listening on Journal Audit Socket.668gitlab # [ 6.397843] systemd[1]: Listening on Console Output Muting Service Socket.669gitlab # [ 6.399533] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.670gitlab # [ 6.400545] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os671gitlab # [ 6.401488] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki672gitlab # [ 6.408870] systemd[1]: Listening on Disk Repartitioning Service Socket.673gitlab # [ 6.410149] systemd[1]: Listening on udev Control Socket.674gitlab # [ 6.411686] systemd[1]: Listening on udev Varlink Socket.675gitlab # [ 6.423067] systemd[1]: Mounting Huge Pages File System...676gitlab # [ 6.429544] systemd[1]: Mounting POSIX Message Queue File System...677gitlab # [ 6.435185] systemd[1]: Mounting Kernel Debug File System...678gitlab # [ 6.442452] systemd[1]: Mounting Kernel Trace File System...679gitlab # [ 6.458169] systemd[1]: Starting Create List of Static Device Nodes...680gitlab # [ 6.466061] systemd[1]: Starting Load Kernel Module 9pnet_virtio...681gitlab # [ 6.466971] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs682gitlab # [ 6.480724] systemd[1]: Mounting Kernel Configuration File System...683gitlab # [ 6.482061] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm684gitlab # [ 6.489473] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore685gitlab # [ 6.505139] systemd[1]: Starting Load Kernel Module fuse...686gitlab # [ 6.506100] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67687gitlab # [ 6.526083] systemd[1]: Starting Journal Service...688gitlab # [ 6.538129] systemd[1]: Starting Load Kernel Modules...689gitlab # [ 6.549176] fuse: init (API version 7.45)690gitlab # [ 6.550001] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...691gitlab # [ 6.563371] systemd[1]: Starting Remount Root and Kernel File Systems...692gitlab # [ 6.564278] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os693gitlab # [ 6.573949] systemd[1]: Starting Coldplug All udev Devices...694gitlab # [ 6.584012] systemd[1]: Listening on Journal Log Access Socket.695gitlab # [ 6.584940] systemd[1]: Mounted Huge Pages File System.696gitlab # [ 6.585800] systemd[1]: Mounted POSIX Message Queue File System.697gitlab # [ 6.586631] systemd[1]: Mounted Kernel Debug File System.698gitlab # [ 6.587504] systemd[1]: Mounted Kernel Trace File System.699gitlab # [ 6.590816] systemd-journald[318]: Collecting audit messages is enabled.700gitlab # [ 6.602640] systemd[1]: Finished Create List of Static Device Nodes.701gitlab # [ 6.606673] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.702gitlab # [ 6.607590] systemd[1]: Finished Load Kernel Module 9pnet_virtio.703gitlab # [ 6.608367] systemd[1]: Mounted Kernel Configuration File System.704gitlab # [ 6.609316] systemd[1]: modprobe@fuse.service: Deactivated successfully.705gitlab # [ 6.610297] systemd[1]: Finished Load Kernel Module fuse.706gitlab # [ 6.611625] systemd[1]: Finished Load Kernel Modules.707gitlab # [ 6.621922] systemd[1]: Mounting FUSE Control File System...708gitlab # [ 6.615424] systemd[1]: Queued start job for default target Multi-User System.[ 6.630916] systemd[1]: Starting Firewall...709gitlab # 710gitlab # [ 6.630064] systemd[1]: Unnecessary job was removed for /dev/vdb.711gitlab # [ 6.631242] systemd[1]: systemd-journald.service: Deactivated successfully.712gitlab # [ 6.632615] systemd-modules-load[319]: Using 3 probe threads713gitlab # [ 6.634199] systemd-modules-load[319]: Module 'atkbd' is built in714gitlab # [ 6.640951] systemd[1]: Starting Apply Kernel Variables...715gitlab # [ 6.636067] systemd-modules-load[319]: Module 'loop' is built in716gitlab # [ 6.640910] systemd-modules-load[319]: Inserted module 'tls'717gitlab # [ 6.647626] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...718gitlab # [ 6.652690] systemd[1]: Started Journal Service.719gitlab # [ 6.653177] EXT4-fs (vda): re-mounted bccaf804-7215-4b9a-b488-51c2452cc900.720gitlab # [ 6.666040] systemd[1]: Finished Remount Root and Kernel File Systems.721gitlab # [ 6.671931] systemd[1]: Mounted FUSE Control File System.722gitlab # [ 6.674703] systemd-oomd[321]: No swap; memory pressure usage will be degraded723gitlab # [ 6.679857] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.724gitlab # [ 6.682620] systemd[1]: Listening on Disk Image Download Service Socket.725gitlab # [ 6.686342] systemd[1]: Starting Flush Journal to Persistent Storage...726gitlab # [ 6.687469] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore727gitlab # [ 6.706807] systemd[1]: Starting Load/Save OS Random Seed...728gitlab # [ 6.707935] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os729gitlab # [ 6.713344] systemd[1]: Finished Apply Kernel Variables.730gitlab # [ 6.722369] systemd-journald[318]: Received client request to flush runtime journal.731gitlab # [ 6.751970] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.732gitlab # [ 6.755323] systemd[1]: Starting Create Static Device Nodes in /dev...733gitlab # [ 6.756898] systemd[1]: Finished Load/Save OS Random Seed.734gitlab # [ 6.758766] systemd[1]: Reached target First Boot Complete.735gitlab # [ 6.759744] systemd[1]: Finished Create Static Device Nodes in /dev.736gitlab # [ 6.761587] systemd[1]: Reached target Preparation for Local File Systems.737gitlab # [ 6.762827] systemd[1]: Starting Rule-based Manager for Device Events and Files...738gitlab # [ 6.764194] systemd[1]: Finished Flush Journal to Persistent Storage.739gitlab # [ 6.815812] systemd-udevd[366]: Using default interface naming scheme 'v261'.740gitlab # [ 6.861602] systemd[1]: Started Rule-based Manager for Device Events and Files.741gitlab # [ 7.246399] systemd[1]: Finished Firewall.742gitlab # [ 7.314984] systemd[1]: Mounting /run/wrappers...743gitlab # [ 7.378199] systemd[1]: Mounted /run/wrappers.744gitlab # [ 7.380576] systemd[1]: Reached target Local File Systems.745gitlab # [ 7.383217] systemd[1]: Listening on Boot Loader Control Service Socket.746gitlab # [ 7.387881] systemd[1]: Starting register-nix-paths.service...747gitlab # [ 7.395447] systemd[1]: Starting Create SUID/SGID Wrappers...748gitlab # [ 7.399134] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.749gitlab # [ 7.417091] systemd[1]: Starting Save Transient machine-id to Disk...750gitlab # [ 7.432213] systemd[1]: Starting Create System Files and Directories...751gitlab # [ 7.474841] systemd[1]: Finished Create System Files and Directories.752gitlab # [ 7.496409] systemd[1]: Starting Rebuild Journal Catalog...753gitlab # [ 7.500127] systemd[1]: Starting Record System Boot/Shutdown in UTMP...754gitlab # [ 7.509910] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.755gitlab # [ 7.523027] systemd[1]: Finished Save Transient machine-id to Disk.756gitlab # [ 7.544522] systemd[1]: Finished Record System Boot/Shutdown in UTMP.757gitlab # [ 7.560304] systemd[1]: Finished Rebuild Journal Catalog.758gitlab # [ 7.563499] systemd[1]: Starting Update is Completed...759gitlab # [ 7.595653] systemd[1]: Finished Update is Completed.760gitlab # [ 7.598298] systemd[1]: Finished Coldplug All udev Devices.761gitlab # [ 7.654911] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs762gitlab # [ 7.665179] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.763gitlab # [ 7.667098] systemd[1]: Finished Create SUID/SGID Wrappers.764gitlab # [ 7.668398] systemd[1]: suid-sgid-wrappers.service: Consumed 166ms CPU time over 270ms wall clock time, 6.2M memory peak, 1.9M read from disk.765gitlab # [ 7.716958] (udev-worker)[410]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.766gitlab # [ 7.719955] (udev-worker)[410]: Network interface NamePolicy= disabled on kernel command line.767gitlab # [ 7.730723] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.768gitlab # [ 7.740767] systemd[1]: Condition check resulted in Virtio network device being skipped.769gitlab # [ 7.742189] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore770gitlab # [ 7.745770] systemd[1]: Starting Create SUID/SGID Wrappers...771gitlab # [ 7.746954] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.772gitlab # [ 7.748808] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67773gitlab # [ 7.751465] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore774gitlab # [ 7.753745] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os775gitlab # [ 7.755224] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os776gitlab # [ 7.756939] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped.777gitlab # [ 7.759627] (udev-worker)[416]: Network interface NamePolicy= disabled on kernel command line.778gitlab # [ 7.872139] mousedev: PS/2 mouse device common for all mice779gitlab # [ 7.903544] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.780gitlab # [ 7.906296] systemd[1]: Finished Create SUID/SGID Wrappers.781gitlab # [ 7.938067] systemd[1]: Finished register-nix-paths.service.782gitlab # [ 7.938986] systemd[1]: Reached target System Initialization.783gitlab # [ 7.939838] systemd[1]: Started Discard unused filesystem blocks once a week.784gitlab # [ 7.940999] systemd[1]: Started logrotate.timer.785gitlab # [ 7.941703] systemd[1]: Started Daily Cleanup of Temporary Directories.786gitlab # [ 7.942620] systemd[1]: Reached target Timer Units.787gitlab # [ 7.943409] systemd[1]: Listening on D-Bus System Message Bus Socket.788gitlab # [ 7.944781] systemd[1]: Listening on Nix Daemon Socket.789gitlab # [ 7.957827] systemd[1]: Starting nixbot CI service socket...790gitlab # [ 7.958792] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.791gitlab # [ 7.959957] systemd[1]: Listening on nixbot CI service socket.792gitlab # [ 7.961335] systemd[1]: Reached target Socket Units.793gitlab # [ 7.963368] systemd[1]: Starting D-Bus System Message Bus...794gitlab # [ 8.000577] dbus-broker-launch[621]: Looking up NSS user entry for 'systemd-timesync'...795gitlab # [ 8.002193] dbus-broker-launch[621]: NSS returned no entry for 'systemd-timesync'796gitlab # [ 8.003506] dbus-broker-launch[621]: Invalid user-name in /nix/store/9zcyylhndlnizbwn48627991zr1jxg4p-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"797gitlab # [ 8.006470] systemd[1]: Started D-Bus System Message Bus.798gitlab # [ 8.018250] dbus-broker-launch[621]: Ready799gitlab # [ 8.019711] systemd[1]: Reached target Basic System.800gitlab # [ 8.021765] systemd[1]: Started backdoor.service.801gitlab # [ 8.027879] systemd[1]: Starting gitlab-config.service...802gitlab # [ 8.048482] systemd[1]: Starting Import lastlog data into lastlog2 database...803gitlab # [ 8.057364] systemd[1]: Starting Logrotate configuration check...804gitlab # [ 8.058976] systemd[1]: Starting Name Service Cache Daemon (nsncd)...805gitlab # connecting to host...806gitlab # [ 8.061497] systemd[1]: Starting Post-Boot Actions...807gitlab # [ 8.071840] systemd[1]: Started Reset console on configuration changes.808gitlab # [ 8.074037] systemd[1]: Starting resolvconf update...809gitlab # [ 8.075230] logrotate[626]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want.810gitlab # [ 8.080371] logrotate[626]: reading config file /nix/store/22afkmbkaxyx50bj9lbkkpmz7pd69hsf-logrotate.conf811gitlab # [ 8.081874] logrotate[626]: note: 'monthly' overrides previously specified 'weekly'812gitlab: Guest shell says: b'Spawning backdoor root shell...\n'813gitlab # [ 8.082857] logrotate[626]: Reading state from file: /var/lib/logrotate.status814gitlab # [ 8.088249] logrotate[626]: state file /var/lib/logrotate.status does not exist815gitlab: connected to guest root shell816gitlab # [ 8.090126] logrotate[626]: Allocating hash table for state file, size 64 entries817gitlab: (connecting took 8.55 seconds)818gitlab # [ 8.091112] logrotate[626]: Handling 4 logs819gitlab: (finished: waiting for the VM to finish booting, in 8.55 seconds)820gitlab # [ 8.093914] logrotate[626]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed821gitlab # [ 8.095966] logrotate[626]: considering log /var/log/btmp822gitlab # [ 8.100096] logrotate[626]: Creating new state823gitlab # [ 8.100780] logrotate[626]: Now: 2026-09-13 10:10824gitlab # [ 8.101481] logrotate[626]: Last rotated at 2026-09-13 10:00825gitlab # [ 8.104722] logrotate[626]: log does not need rotating (log has already been rotated)826gitlab # [ 8.105949] logrotate[626]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed827gitlab # [ 8.112084] logrotate[626]: considering log /var/log/wtmp828gitlab # [ 8.112883] logrotate[626]: Creating new state829gitlab # [ 8.113521] logrotate[626]: Now: 2026-09-13 10:10830gitlab # [ 8.114650] logrotate[626]: Last rotated at 2026-09-13 10:00831gitlab # [ 8.115607] logrotate[626]: log does not need rotating (log has already been rotated)832gitlab # [ 8.119561] logrotate[626]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed833gitlab # [ 8.121600] logrotate[626]: switching euid from 0 to 165 and egid from 0 to 165 (pid 626)834gitlab # [ 8.122649] logrotate[626]: considering log /var/gitlab/state/log/*.log835gitlab # [ 8.123648] logrotate[626]: log /var/gitlab/state/log/*.log does not exist -- skipping836gitlab # [ 8.124967] logrotate[626]: Creating new state837gitlab # [ 8.125660] logrotate[626]: switching euid from 165 to 0 and egid from 165 to 0 (pid 626)838gitlab # [ 8.126806] logrotate[626]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed839gitlab # [ 8.131445] logrotate[626]: switching euid from 0 to 60 and egid from 0 to 60 (pid 626)840gitlab # [ 8.133698] logrotate[626]: considering log /var/log/nginx/*.log841gitlab # [ 8.134580] logrotate[626]: log /var/log/nginx/*.log does not exist -- skipping842gitlab # [ 8.135555] logrotate[626]: Creating new state843gitlab # [ 8.137154] logrotate[626]: not running postrotate script, since no logs were rotated844gitlab # [ 8.140154] logrotate[626]: switching euid from 60 to 0 and egid from 60 to 0 (pid 626)845gitlab # [ 8.142448] systemd[1]: Finished Logrotate configuration check.846gitlab # [ 8.143709] nsncd[629]: Sep 13 10:10:50.027 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"847gitlab # [ 8.147773] systemd[1]: Finished Post-Boot Actions.848gitlab # [ 8.149988] systemd[1]: Finished Import lastlog data into lastlog2 database.849gitlab # [ 8.151098] systemd[1]: Started Name Service Cache Daemon (nsncd).850gitlab # [ 8.153094] systemd[1]: Reached target Host and Network Name Lookups.851gitlab # [ 8.154156] systemd[1]: Reached target User and Group Name Lookups.852gitlab # [ 8.155138] systemd[1]: Starting User Login Management...853gitlab # [ 8.205612] systemd-logind[664]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard)854gitlab # [ 8.208143] systemd-logind[664]: Watching system buttons on /dev/input/event0 (gpio-keys)855gitlab # [ 8.209283] systemd-logind[664]: New seat seat0.856gitlab # [ 8.210529] systemd[1]: Started User Login Management.857gitlab # [ 8.213094] systemd[1]: Starting linger-users.service...858gitlab # [ 8.240565] systemd[1]: linger-users.service: Deactivated successfully.859gitlab # [ 8.241585] systemd[1]: Finished linger-users.service.860gitlab # [ 8.244482] systemd[1]: Stopped target Host and Network Name Lookups.861gitlab # [ 8.247355] systemd[1]: Stopping Host and Network Name Lookups...862gitlab # [ 8.250001] systemd[1]: Stopped target User and Group Name Lookups.863gitlab # [ 8.251009] systemd[1]: Stopping User and Group Name Lookups...864gitlab # [ 8.254161] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...865gitlab # [ 8.255233] systemd[1]: nscd.service: Deactivated successfully.866gitlab # [ 8.256261] systemd[1]: Stopped Name Service Cache Daemon (nsncd).867gitlab # [ 8.257160] systemd[1]: nscd.service: Consumed 27ms CPU time over 190ms wall clock time, 7.4M memory peak, 5.8M read from disk.868gitlab # [ 8.262831] systemd[1]: Starting Name Service Cache Daemon (nsncd)...869gitlab # [ 8.284926] systemd[1]: Finished resolvconf update.870gitlab # [ 8.287567] systemd[1]: Reached target Preparation for Network.871gitlab # [ 8.292157] systemd[1]: Starting DHCP Client...872gitlab # [ 8.293246] nsncd[721]: Sep 13 10:10:50.201 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"873gitlab # [ 8.301016] systemd[1]: Starting Address configuration of eth1...874gitlab # [ 8.304831] systemd[1]: Starting Extra networking commands....875gitlab # [ 8.307563] systemd[1]: Started Name Service Cache Daemon (nsncd).876gitlab # [ 8.309022] systemd[1]: Reached target Host and Network Name Lookups.877gitlab # [ 8.310028] systemd[1]: Reached target User and Group Name Lookups.878gitlab # [ 8.337275] network-addresses-eth1-start[745]: adding address 192.168.1.1/24... done879gitlab # [ 8.344630] network-addresses-eth1-start[745]: adding address 2001:db8:1::1/64... done880gitlab # [ 8.354346] systemd[1]: Finished Address configuration of eth1.881gitlab # [ 8.418025] dhcpcd[761]: dhcpcd-10.3.2 starting882gitlab # [ 8.423661] systemd[1]: Finished Extra networking commands..883gitlab # [ 8.425054] dhcpcd[809]: dev: loaded udev884gitlab # [ 8.425714] systemd[1]: Reached target Network.885gitlab # [ 8.429397] systemd[1]: Starting gitlab-workhorse.service...886gitlab # [ 8.431228] systemd[1]: Starting Nginx Web Server...887gitlab # [ 8.452576] systemd[1]: Starting PostgreSQL Server...888gitlab # [ 8.473823] systemd[1]: Starting Redis Server - redis-gitlab...889gitlab # [ 8.500504] systemd[1]: Starting Permit User Sessions...890gitlab # [ 8.528686] 8021q: 802.1Q VLAN Support v1.8891gitlab # [ 8.529100] 8021q: adding VLAN 0 to HW filter on device eth1892gitlab # [ 8.536756] systemd[1]: Finished Permit User Sessions.893gitlab # [ 8.551662] systemd[1]: Started Getty on tty1.894gitlab # [ 8.553681] systemd[1]: Reached target Login Prompts.895gitlab # [ 8.579681] cfg80211: Loading compiled-in X.509 certificates for regulatory database896gitlab # [ 8.593148] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'897gitlab # [ 8.593668] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'898gitlab # [ 8.596451] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2899gitlab # [ 8.596774] cfg80211: failed to load regulatory.db900gitlab # [ 8.605697] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:09.0/virtio8/input/input3901gitlab # [ 8.670017] 8021q: adding VLAN 0 to HW filter on device eth0902gitlab # [ 8.665897] dhcpcd[809]: eth0: waiting for carrier903gitlab # [ 8.667043] dhcpcd[809]: libudev: received NULL device904gitlab # [ 8.667758] dhcpcd[809]: libudev: received NULL device905gitlab # [ 8.671679] dhcpcd[809]: eth0: carrier acquired906gitlab # [ 8.674924] dhcpcd[809]: DUID 00:01:00:01:32:39:33:2a:52:54:00:12:34:56907gitlab # [ 8.675932] dhcpcd[809]: eth0: IAID 00:12:34:56908gitlab # [ 8.676701] dhcpcd[809]: eth0: adding address fe80::5054:ff:fe12:3456909gitlab # [ 8.693571] nginx-pre-start[837]: nginx: the configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf syntax is ok910gitlab # [ 8.696690] nginx-pre-start[837]: nginx: configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf test is successful911gitlab # [ 8.702887] systemd[1]: Started Nginx Web Server.912gitlab # [ 8.715218] postgresql-pre-start[839]: The files belonging to this database system will be owned by user "postgres".913gitlab # [ 8.723763] postgresql-pre-start[839]: This user must also own the server process.914gitlab # [ 8.724902] postgresql-pre-start[839]: The database cluster will be initialized with locale "en_US.UTF-8".915gitlab # [ 8.726130] postgresql-pre-start[839]: The default database encoding has accordingly been set to "UTF8".916gitlab # [ 8.727472] postgresql-pre-start[839]: The default text search configuration will be set to "english".917gitlab # [ 8.736224] postgresql-pre-start[839]: Data page checksums are enabled.918gitlab # [ 8.737415] postgresql-pre-start[839]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok919gitlab # [ 8.738911] postgresql-pre-start[839]: creating subdirectories ... ok920gitlab # [ 8.739865] postgresql-pre-start[839]: selecting dynamic shared memory implementation ... posix921gitlab # [ 8.762789] redis-server[841]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.922gitlab # [ 8.765917] redis-server[841]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo923gitlab # [ 8.768744] redis-server[841]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=841, just started924gitlab # [ 8.770236] systemd[1]: Started Redis Server - redis-gitlab.925gitlab # [ 8.771021] redis-server[841]: Configuration loaded926gitlab # [ 8.775464] redis[841]: monotonic clock: ARM CNTVCT @ 1000 ticks/us927gitlab # [ 8.776580] redis[841]: Running mode=standalone, port=0.928gitlab # [ 8.777523] redis[841]: Server initialized929gitlab # [ 8.778268] redis[841]: Ready to accept connections unix930gitlab # [ 8.779136] redis[841]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.931gitlab # [ 8.784195] postgresql-pre-start[839]: selecting default "max_connections" ... 100932gitlab # [ 8.829520] postgresql-pre-start[839]: selecting default "shared_buffers" ... 128MB933gitlab # [ 8.928084] systemd[1]: Starting Virtual Console Setup...934gitlab # [ 8.961544] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.935gitlab # [ 8.962647] systemd[1]: Stopped Virtual Console Setup.936gitlab # [ 8.979356] systemd[1]: Starting Virtual Console Setup...937gitlab # [ 8.991753] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.938gitlab # [ 9.043071] systemd-logind[664]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard)939gitlab # [ 9.046846] systemd-vconsole-setup[876]: Configuration of first virtual console was skipped, ignoring remaining ones.940gitlab # [ 9.051823] systemd[1]: Finished Virtual Console Setup.941gitlab # [ 9.221275] systemd[1]: Started gitlab-workhorse.service.942gitlab # [ 9.367791] postgresql-pre-start[839]: selecting default time zone ... UTC943gitlab # [ 9.372061] postgresql-pre-start[839]: creating configuration files ... ok944gitlab # [ 9.504553] gitlab-workhorse[885]: time="2026-09-13T10:10:51Z" level=info msg=Starting build_time=19700101.000000 version=19.0.4945gitlab # [ 9.506187] gitlab-workhorse[885]: time="2026-09-13T10:10:51Z" level=info msg="Using redis/go-redis"946gitlab # [ 9.507389] gitlab-workhorse[885]: time="2026-09-13T10:10:51Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix947gitlab # [ 9.624909] postgresql-pre-start[839]: running bootstrap script ... ok948gitlab # [ 9.901119] dhcpcd[809]: eth0: soliciting an IPv6 router949gitlab # [ 9.903558] dhcpcd[809]: eth0: Router Advertisement from fe80::2950gitlab # [ 9.906128] dhcpcd[809]: eth0: adding address fec0::5054:ff:fe12:3456/64951gitlab # [ 9.909086] dhcpcd[809]: eth0: adding route to fec0::/64952gitlab # [ 9.911365] dhcpcd[809]: eth0: adding default route via fe80::2953gitlab # [ 9.933421] dhcpcd[809]: eth0: soliciting a DHCP lease954gitlab # [ 9.948970] dhcpcd[809]: eth0: offered 10.0.2.15 from 10.0.2.2955gitlab # [ 9.964487] dhcpcd[809]: eth0: probing address 10.0.2.15/24956gitlab # [ 10.324783] postgresql-pre-start[839]: performing post-bootstrap initialization ... ok957gitlab # [ 10.521074] postgresql-pre-start[839]: syncing data to disk ... ok958gitlab # [ 10.524111] postgresql-pre-start[839]: initdb: warning: enabling "trust" authentication for local connections959gitlab # [ 10.528223] postgresql-pre-start[839]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.960gitlab # [ 10.534956] postgresql-pre-start[839]: Success. You can now start the database server using:961gitlab # [ 10.538614] postgresql-pre-start[839]: pg_ctl -D /var/lib/postgresql/18 -l logfile start962gitlab # [ 10.670983] postgres[914]: [914] LOG: starting PostgreSQL 18.4 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit963gitlab # [ 10.676825] postgres[914]: [914] LOG: listening on IPv6 address "::1", port 5432964gitlab # [ 10.680377] postgres[914]: [914] LOG: listening on IPv4 address "127.0.0.1", port 5432965gitlab # [ 10.684852] postgres[914]: [914] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"966gitlab # [ 10.690489] postgres[924]: [924] LOG: database system was shut down at 2026-09-13 10:10:52 GMT967gitlab # [ 10.694339] postgres[914]: [914] LOG: database system is ready to accept connections968gitlab # [ 10.699991] systemd[1]: Started PostgreSQL Server.969gitlab # [ 10.703440] systemd[1]: Starting PostgreSQL Setup Scripts...970gitlab # [ 11.146232] postgresql-setup-start[935]: CREATE DATABASE971gitlab # [ 11.159401] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[940]: mkdir -p /var/gitlab/state/home/.ssh: OK972gitlab # [ 11.162063] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[940]: chmod 700 /var/gitlab/state/home/.ssh: OK973gitlab # [ 11.199085] postgresql-setup-start[950]: CREATE ROLE974gitlab # [ 11.217345] postgresql-setup-start[954]: ALTER DATABASE975gitlab # [ 11.263735] systemd[1]: Finished gitlab-config.service.976gitlab # [ 11.266821] postgresql-setup-start[963]: CREATE ROLE977gitlab # [ 11.271341] systemd[1]: Started gitaly.service.978gitlab # [ 11.273623] systemd[1]: Finished PostgreSQL Setup Scripts.979gitlab # [ 11.276515] systemd[1]: Reached target PostgreSQL.980gitlab # [ 11.281750] systemd[1]: Starting gitlab-db-config.service...981gitlab # [ 11.288158] systemd[1]: Starting gitlab-postgresql.service...982gitlab # [ 11.491440] gitlab-postgresql-start[976]: CREATE DATABASE983gitlab # [ 11.543144] gitlab-postgresql-start[1001]: CREATE EXTENSION984gitlab # [ 11.651653] gitlab-postgresql-start[1003]: CREATE EXTENSION985gitlab # [ 11.655908] systemd[1]: Finished gitlab-postgresql.service.986gitlab # [ 11.855846] gitaly[966]: time="2026-09-13T10:10:53.764Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/0r0qd84zrxhqq373hk9spv8j9pg0qlnq-gitaly.toml' and fix the errors that are printed." pid=966987gitlab # [ 11.862778] gitaly[966]: time="2026-09-13T10:10:53.764Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=966988gitlab # [ 11.865936] gitaly[966]: time="2026-09-13T10:10:53.764Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=966989gitlab # [ 11.869548] gitaly[966]: time="2026-09-13T10:10:53.764Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=966990gitlab # [ 11.872722] gitaly[966]: time="2026-09-13T10:10:53.764Z" level=info msg="Starting Gitaly" pid=966 version=991gitlab # [ 12.052873] gitaly[966]: time="2026-09-13T10:10:53.961Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=196 pid=966992gitlab # [ 12.060266] gitaly[966]: time="2026-09-13T10:10:53.961Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=966993gitlab # [ 12.061887] gitaly[966]: time="2026-09-13T10:10:53.962Z" level=info msg="finished initializing command factory" duration_ms=0 pid=966994gitlab # [ 12.063396] gitaly[966]: time="2026-09-13T10:10:53.962Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-966/git-exec-2747212880.d/git pid=966995gitlab # [ 12.071119] gitaly[966]: time="2026-09-13T10:10:53.978Z" level=info msg="finished detecting git version" duration_ms=16 pid=966996gitlab # [ 12.072791] gitaly[966]: time="2026-09-13T10:10:53.978Z" level=info msg="using Git version" pid=966 version=2.53.g997gitlab # [ 12.074118] gitaly[966]: time="2026-09-13T10:10:53.979Z" level=info msg="clearing disk cache object folder" pid=966 storage=default998gitlab # [ 12.075594] gitaly[966]: time="2026-09-13T10:10:53.979Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3990392948 pid=966 storage=default999gitlab # [ 12.078046] gitaly[966]: time="2026-09-13T10:10:53.979Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3990392948 pid=966 storage=default1000gitlab # [ 12.080696] gitaly[966]: time="2026-09-13T10:10:53.979Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=9661001gitlab # [ 12.083045] gitaly[966]: time="2026-09-13T10:10:53.979Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=9661002gitlab # [ 12.085529] gitaly[966]: time="2026-09-13T10:10:53.979Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3990392948 clear_duration_ms=0 pid=966 storage=default1003gitlab # [ 12.106961] gitaly[966]: time="2026-09-13T10:10:54.015Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=9661004gitlab # [ 12.109196] gitaly[966]: time="2026-09-13T10:10:54.016Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=966 schema=unix1005gitlab # [ 12.111072] gitaly[966]: time="2026-09-13T10:10:54.016Z" level=info msg="listening at address" address=/run/gitaly/gitaly-966/sock.d/intern pid=966 schema=unix1006gitlab # [ 12.112919] gitaly[966]: time="2026-09-13T10:10:54.015Z" level=info msg="counting repositories" pid=9661007gitlab # [ 12.114120] gitaly[966]: time="2026-09-13T10:10:54.016Z" level=info msg="starting to count repositories" pid=966 storage_path=/var/gitlab/state/repositories1008gitlab # [ 12.115839] gitaly[966]: time="2026-09-13T10:10:54.016Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=966 storage_path=/var/gitlab/state/repositories1009gitlab # [ 12.118014] gitaly[966]: time="2026-09-13T10:10:54.016Z" level=info msg="completed counting all repositories" counting_duration=0 pid=9661010gitlab # [ 12.119546] gitaly[966]: time="2026-09-13T10:10:54.018Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=9661011gitlab # [ 12.121291] gitaly[966]: time="2026-09-13T10:10:54.018Z" level=info msg="Started Gitaly" duration_ms=253 pid=9661012gitlab # [ 12.122540] gitaly[966]: time="2026-09-13T10:10:54.018Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=9661013gitlab # [ 12.124021] gitaly[966]: time="2026-09-13T10:10:54.018Z" level=info msg="maintenance: daily scheduled" pid=966 scheduled="2026-09-13 12:00:00 +0000 UTC"1014gitlab # [ 12.125697] gitaly[966]: time="2026-09-13T10:10:54.029Z" level=info msg="finished tempdir cleaner walk" pid=966 storage=default time_ms=131015gitlab # [ 12.610391] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[971]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1016gitlab # [ 12.620758] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[971]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1017gitlab # [ 12.630792] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[971]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1018gitlab # [ 14.837162] dhcpcd[809]: eth0: leased 10.0.2.15 for 86400 seconds1019gitlab # [ 14.837487] dhcpcd[809]: eth0: adding route to 10.0.2.0/241020gitlab # [ 14.837614] dhcpcd[809]: eth0: adding default route via 10.0.2.21021gitlab # [ 14.974401] systemd[1]: Started DHCP Client.1022gitlab # [ 14.976376] systemd[1]: Reached target Network is Online.1023gitlab # [ 15.654077] gitaly[966]: time="2026-09-13T10:10:57.562Z" level=info msg="License database preloaded" duration_ms=3797 pid=9661024gitlab # [ 58.512581] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[971]: W, [2026-09-13T10:11:40.420243 #971] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1025gitlab # [ 59.578918] gitlab-workhorse[885]: time="2026-09-13T10:11:41Z" level=error correlation_id= duration_ms=1 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1026gitlab # [ 61.589557] postgres[1506]: [1506] ERROR: relation "feature_gates" does not exist at character 5231027gitlab # [ 61.589941] postgres[1506]: [1506] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod),1028gitlab # [ 61.590073] postgres[1506]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1029gitlab # [ 61.590191] postgres[1506]: c.collname, col_description(a.attrelid, a.attnum) AS comment,1030gitlab # [ 61.590303] postgres[1506]: attidentity AS identity,1031gitlab # [ 61.590439] postgres[1506]: attgenerated as attgenerated1032gitlab # [ 61.590560] postgres[1506]: FROM pg_attribute a1033gitlab # [ 61.590674] postgres[1506]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1034gitlab # [ 61.590792] postgres[1506]: LEFT JOIN pg_type t ON a.atttypid = t.oid1035gitlab # [ 61.590906] postgres[1506]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1036gitlab # [ 61.591002] postgres[1506]: WHERE a.attrelid = '"feature_gates"'::regclass1037gitlab # [ 61.591124] postgres[1506]: AND a.attnum > 0 AND NOT a.attisdropped1038gitlab # [ 61.591254] postgres[1506]: ORDER BY a.attnum1039gitlab # [ 61.591365] postgres[1506]:1040gitlab # [ 66.726208] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1548]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping1041gitlab # [ 66.726700] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1548]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping1042gitlab # [ 69.586989] gitlab-workhorse[885]: time="2026-09-13T10:11:51Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1043gitlab # [ 79.588594] gitlab-workhorse[885]: time="2026-09-13T10:12:01Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1044gitlab # [ 89.598201] gitlab-workhorse[885]: time="2026-09-13T10:12:11Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1045gitlab # [ 94.500561] postgres[1660]: [1660] ERROR: relation "table_without_models" does not exist1046gitlab # [ 94.500925] postgres[1660]: [1660] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1047gitlab # [ 99.606323] gitlab-workhorse[885]: time="2026-09-13T10:12:21Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1048gitlab # [ 109.617154] gitlab-workhorse[885]: time="2026-09-13T10:12:31Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1049gitlab # [ 119.618258] gitlab-workhorse[885]: time="2026-09-13T10:12:41Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1050gitlab # [ 124.848223] systemd[1]: Finished gitlab-db-config.service.1051gitlab # [ 124.854860] systemd[1]: Started gitlab-sidekiq.service.1052gitlab # [ 124.861072] systemd[1]: Starting gitlab.service...1053gitlab # [ 125.446647] bundle[2044]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1054gitlab # [ 125.452976] bundle[2044]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1055gitlab # [ 125.458751] bundle[2044]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1056gitlab # [ 129.626440] gitlab-workhorse[885]: time="2026-09-13T10:12:51Z" level=error correlation_id= duration_ms=5 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1057gitlab # [ 130.205896] bundle[2044]: {"timestamp":"2026-09-13T10:12:52.083Z","pid":2044,"message":"* Enabling systemd notification integration"}1058gitlab # [ 130.210985] bundle[2044]: {"timestamp":"2026-09-13T10:12:52.113Z","pid":2044,"message":"Puma starting in cluster mode..."}1059gitlab # [ 130.215962] bundle[2044]: {"timestamp":"2026-09-13T10:12:52.114Z","pid":2044,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"}1060gitlab # [ 130.222324] bundle[2044]: {"timestamp":"2026-09-13T10:12:52.114Z","pid":2044,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1061gitlab # [ 130.228209] bundle[2044]: {"timestamp":"2026-09-13T10:12:52.114Z","pid":2044,"message":"* Min threads: 0"}1062gitlab # [ 130.231960] bundle[2044]: {"timestamp":"2026-09-13T10:12:52.114Z","pid":2044,"message":"* Max threads: 4"}1063gitlab # [ 130.235773] bundle[2044]: {"timestamp":"2026-09-13T10:12:52.114Z","pid":2044,"message":"* Environment: production"}1064gitlab # [ 130.240150] bundle[2044]: {"timestamp":"2026-09-13T10:12:52.114Z","pid":2044,"message":"* Master PID: 2044"}1065gitlab # [ 130.244051] bundle[2044]: {"timestamp":"2026-09-13T10:12:52.114Z","pid":2044,"message":"* Workers: 2"}1066gitlab # [ 130.247817] bundle[2044]: {"timestamp":"2026-09-13T10:12:52.114Z","pid":2044,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"}1067gitlab # [ 130.252874] bundle[2044]: {"timestamp":"2026-09-13T10:12:52.114Z","pid":2044,"message":"* Preloading application"}1068gitlab # [ 130.257863] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-09-13T10:12:52.093Z","message":"Starting cluster with 1 processes"}1069gitlab # [ 139.639452] gitlab-workhorse[885]: time="2026-09-13T10:13:01Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1070gitlab # [ 149.640570] gitlab-workhorse[885]: time="2026-09-13T10:13:11Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1071gitlab # [ 159.650699] gitlab-workhorse[885]: time="2026-09-13T10:13:21Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1072gitlab # [ 168.732618] bundle[2044]: W, [2026-09-13T10:13:30.640450 #2044] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1073gitlab # [ 169.661469] gitlab-workhorse[885]: time="2026-09-13T10:13:31Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1074gitlab # [ 172.998312] sidekiq-cluster[2093]: 2026-09-13T10:13:34.906Z pid=2093 tid=xol INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2093, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>432697344}1075gitlab # [ 173.290494] sidekiq-cluster[2093]: W, [2026-09-13T10:13:35.199301 #2093] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1076gitlab # [ 173.349054] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:35.257Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1077gitlab # [ 177.166876] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.074Z","message":"GitLab reliable fetch activated!"}1078gitlab # [ 177.167246] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.074Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1079gitlab # [ 177.236991] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.145Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1080gitlab # [ 177.243907] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.152Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"}1081gitlab # [ 177.251014] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.158Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1082gitlab # [ 177.257659] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.164Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1083gitlab # [ 177.265012] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.169Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1084gitlab # [ 177.271725] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.174Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1085gitlab # [ 177.278520] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.178Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1086gitlab # [ 177.288390] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.182Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1087gitlab # [ 177.294136] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.185Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1088gitlab # [ 177.300185] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.195Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1089gitlab # [ 177.306272] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.206Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1090gitlab # [ 177.311703] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.213Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1091gitlab # [ 177.316279] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.217Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1092gitlab # [ 177.320647] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.221Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1093gitlab # [ 177.324798] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.225Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1094gitlab # [ 177.328757] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.229Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1095gitlab # [ 177.332450] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.233Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1096gitlab # [ 177.340106] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.238Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1097gitlab # [ 177.344172] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.242Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1098gitlab # [ 177.347911] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.246Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1099gitlab # [ 177.352315] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.250Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1100gitlab # [ 177.356985] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.253Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1101gitlab # [ 177.361320] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.255Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1102gitlab # [ 177.364858] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.258Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1103gitlab # [ 177.368253] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.261Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1104gitlab # [ 177.372406] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.264Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1105gitlab # [ 177.375310] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.267Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1106gitlab # [ 177.378491] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.270Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1107gitlab # [ 177.381694] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.272Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1108gitlab # [ 177.384957] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.275Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1109gitlab # [ 177.387929] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.277Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1110gitlab # [ 177.391400] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.280Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1111gitlab # [ 177.394417] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.282Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1112gitlab # [ 177.397527] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.285Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1113gitlab # [ 177.400365] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.287Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1114gitlab # [ 177.403105] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.289Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1115gitlab # [ 177.405941] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.291Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1116gitlab # [ 177.408479] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.293Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1117gitlab # [ 177.410899] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.295Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1118gitlab # [ 177.413318] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.297Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1119gitlab # [ 177.415751] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.299Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1120gitlab # [ 177.418213] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.301Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1121gitlab # [ 177.420596] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.303Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1122gitlab # [ 177.422796] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.305Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1123gitlab # [ 177.425233] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.307Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1124gitlab # [ 177.427654] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.309Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1125gitlab # [ 177.430026] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.311Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1126gitlab # [ 177.432279] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.313Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1127gitlab # [ 177.434369] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.316Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1128gitlab # [ 177.436528] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.319Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1129gitlab # [ 177.438597] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.321Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1130gitlab # [ 177.440746] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.324Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1131gitlab # [ 177.442795] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.337Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1132gitlab # [ 177.445201] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.340Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1133gitlab # [ 177.447328] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.342Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1134gitlab # [ 177.449547] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.345Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1135gitlab # [ 177.451615] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.347Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1136gitlab # [ 177.453944] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.350Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1137gitlab # [ 177.456063] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.352Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1138gitlab # [ 177.458169] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.354Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1139gitlab # [ 177.460147] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.356Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1140gitlab # [ 177.462084] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.358Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1141gitlab # [ 177.464149] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.361Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1142gitlab # [ 177.466249] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.363Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1143gitlab # [ 177.468447] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.365Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1144gitlab # [ 177.470546] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.368Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1145gitlab # [ 177.472595] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.370Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1146gitlab # [ 177.474692] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.372Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1147gitlab # [ 177.476984] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.373Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1148gitlab # [ 177.479218] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.375Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1149gitlab # [ 177.481536] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.377Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1150gitlab # [ 177.483698] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.379Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1151gitlab # [ 177.486033] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.381Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1152gitlab # [ 177.488361] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.382Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1153gitlab # [ 177.490622] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.384Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1154gitlab # [ 177.492924] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.386Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1155gitlab # [ 177.495023] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.388Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1156gitlab # [ 177.497468] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.390Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1157gitlab # [ 177.499536] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.392Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1158gitlab # [ 177.501695] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.393Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1159gitlab # [ 177.503998] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.396Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1160gitlab # [ 177.506336] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.397Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1161gitlab # [ 177.508589] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.400Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1162gitlab # [ 177.510716] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.402Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1163gitlab # [ 177.513076] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.404Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1164gitlab # [ 177.515199] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.406Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1165gitlab # [ 177.517589] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.408Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1166gitlab # [ 177.519751] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.410Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1167gitlab # [ 177.522156] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.412Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1168gitlab # [ 177.524427] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.414Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1169gitlab # [ 177.526837] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.416Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1170gitlab # [ 177.529142] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.418Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1171gitlab # [ 177.531370] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.419Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1172gitlab # [ 177.533828] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.421Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1173gitlab # [ 177.535739] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.424Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1174gitlab # [ 177.538153] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.426Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1175gitlab # [ 177.540331] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.428Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1176gitlab # [ 177.542476] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.430Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1177gitlab # [ 177.544710] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.440Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1178gitlab # [ 177.546866] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.442Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1179gitlab # [ 177.549055] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.444Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1180gitlab # [ 177.551238] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.447Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1181gitlab # [ 177.597525] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.505Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1182gitlab # [ 177.600561] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.509Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1183gitlab # [ 177.603488] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.512Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1184gitlab # [ 177.606589] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.514Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1185gitlab # [ 177.609324] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.517Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1186gitlab # [ 177.612509] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.521Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1187gitlab # [ 177.615276] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:13:39.523Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1188gitlab # [ 179.668401] gitlab-workhorse[885]: time="2026-09-13T10:13:41Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1189gitlab # [ 189.671816] gitlab-workhorse[885]: time="2026-09-13T10:13:51Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1190gitlab # [ 199.672867] gitlab-workhorse[885]: time="2026-09-13T10:14:01Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1191gitlab # [ 209.683707] gitlab-workhorse[885]: time="2026-09-13T10:14:11Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1192gitlab # [ 215.662304] postgres[2812]: [2812] ERROR: relation "table_without_models" does not exist1193gitlab # [ 215.662686] postgres[2812]: [2812] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1194gitlab # [ 219.138813] sidekiq-cluster[2093]: InitializerConnections Query: SET client_min_messages TO 'warning'1195gitlab # [ 219.142855] sidekiq-cluster[2093]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value'1196gitlab # [ 219.147382] sidekiq-cluster[2093]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature'1197gitlab # [ 219.151358] sidekiq-cluster[2093]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value'1198gitlab # [ 219.155512] sidekiq-cluster[2093]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?'1199gitlab # [ 219.159252] sidekiq-cluster[2093]: InitializerConnections Backtrace: lib/feature.rb:156:in `disabled?'1200gitlab # [ 219.163020] sidekiq-cluster[2093]: InitializerConnections Backtrace: lib/gitlab/background_task.rb:60:in `block (2 levels) in start'1201gitlab # [ 219.167418] sidekiq-cluster[2093]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1202gitlab # [ 219.171592] sidekiq-cluster[2093]: InitializerConnections Query: SET standard_conforming_strings = on1203gitlab # [ 219.174520] sidekiq-cluster[2093]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value'1204gitlab # [ 219.178089] sidekiq-cluster[2093]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature'1205gitlab # [ 219.180952] sidekiq-cluster[2093]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value'1206gitlab # [ 219.183794] sidekiq-cluster[2093]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?'1207gitlab # [ 219.186327] sidekiq-cluster[2093]: InitializerConnections Backtrace: lib/feature.rb:156:in `disabled?'1208gitlab # [ 219.188746] sidekiq-cluster[2093]: InitializerConnections Backtrace: lib/gitlab/background_task.rb:60:in `block (2 levels) in start'1209gitlab # [ 219.191620] sidekiq-cluster[2093]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1210gitlab # [ 219.194553] sidekiq-cluster[2093]: InitializerConnections Query: SET intervalstyle = iso_86011211gitlab # [ 219.196536] sidekiq-cluster[2093]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value'1212gitlab # [ 219.198918] sidekiq-cluster[2093]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature'1213gitlab # [ 219.200996] sidekiq-cluster[2093]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value'1214gitlab # [ 219.203080] sidekiq-cluster[2093]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?'1215gitlab # [ 219.205317] sidekiq-cluster[2093]: InitializerConnections Backtrace: lib/feature.rb:156:in `disabled?'1216gitlab # [ 219.207123] sidekiq-cluster[2093]: InitializerConnections Backtrace: lib/gitlab/background_task.rb:60:in `block (2 levels) in start'1217gitlab # [ 219.209417] sidekiq-cluster[2093]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1218gitlab # [ 219.342058] sidekiq-cluster[2093]: InitializerConnections Query: /*application:sidekiq,db_config_database:gitlab,db_config_name:main*/ SELECT t.oid, t.typname1219gitlab # [ 219.347846] sidekiq-cluster[2093]: FROM pg_type as t1220gitlab # [ 219.350041] sidekiq-cluster[2093]: WHERE t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'numeric', 'bool', 'timestamp', 'timestamptz')1221gitlab # [ 219.355345] sidekiq-cluster[2093]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value'1222gitlab # [ 219.359975] sidekiq-cluster[2093]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature'1223gitlab # [ 219.363708] sidekiq-cluster[2093]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value'1224gitlab # [ 219.367492] sidekiq-cluster[2093]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?'1225gitlab # [ 219.370594] sidekiq-cluster[2093]: InitializerConnections Backtrace: lib/feature.rb:156:in `disabled?'1226gitlab # [ 219.373534] sidekiq-cluster[2093]: InitializerConnections Backtrace: lib/gitlab/background_task.rb:60:in `block (2 levels) in start'1227gitlab # [ 219.376983] sidekiq-cluster[2093]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1228gitlab # [ 219.380374] sidekiq-cluster[2093]: InitializerConnections Query: SET SESSION timezone TO 'UTC'1229gitlab # [ 219.382626] sidekiq-cluster[2093]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value'1230gitlab # [ 219.385393] sidekiq-cluster[2093]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature'1231gitlab # [ 219.387676] sidekiq-cluster[2093]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value'1232gitlab # [ 219.390119] sidekiq-cluster[2093]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?'1233gitlab # [ 219.392197] sidekiq-cluster[2093]: InitializerConnections Backtrace: lib/feature.rb:156:in `disabled?'1234gitlab # [ 219.394203] sidekiq-cluster[2093]: InitializerConnections Backtrace: lib/gitlab/background_task.rb:60:in `block (2 levels) in start'1235gitlab # [ 219.396729] sidekiq-cluster[2093]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1236gitlab # [ 219.399293] sidekiq-cluster[2093]: InitializerConnections Query: /*application:sidekiq,db_config_database:gitlab,db_config_name:main*/ SELECT "feature_gates"."key", "feature_gates"."value" FROM "feature_gates" WHERE "feature_gates"."feature_key" = 'sidekiq_throttling_middleware'1237gitlab # [ 219.404181] sidekiq-cluster[2093]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value'1238gitlab # [ 219.406232] sidekiq-cluster[2093]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature'1239gitlab # [ 219.407997] sidekiq-cluster[2093]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value'1240gitlab # [ 219.409863] sidekiq-cluster[2093]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?'1241gitlab # [ 219.411510] sidekiq-cluster[2093]: InitializerConnections Backtrace: lib/feature.rb:156:in `disabled?'1242gitlab # [ 219.413180] sidekiq-cluster[2093]: InitializerConnections Backtrace: lib/gitlab/background_task.rb:60:in `block (2 levels) in start'1243gitlab # [ 219.415191] sidekiq-cluster[2093]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1244gitlab # [ 219.690294] gitlab-workhorse[885]: time="2026-09-13T10:14:21Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1245gitlab # [ 221.194350] postgres[2851]: [2851] ERROR: relation "table_without_models" does not exist1246gitlab # [ 221.194688] postgres[2851]: [2851] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1247gitlab # [ 222.051374] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:23.955Z","message":"Booted Rails 7.2.3.1 application in production environment"}1248gitlab # [ 222.058031] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:23.962Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1249gitlab # [ 222.064543] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:23.962Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1250gitlab # [ 222.069877] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:23.962Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1251gitlab # [ 222.075907] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:23.965Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1252gitlab # [ 224.763577] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:26.671Z","message":"Cleaning working queues"}1253gitlab # [ 224.768371] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:26.673Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"027842b6225d66517d833de8","created_at":"2026-09-13T10:12:46.189Z","trace_propagation_headers":{"sentry-trace":"dd67c4bd170847b0aa02ab8c655a70ec-5dce2b7c0486457e","baggage":"sentry-trace_id=dd67c4bd170847b0aa02ab8c655a70ec,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"87c9b4b8df264301766009d96ab47e62","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-13T10:12:46.202Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-027842b6225d66517d833de8: start","job_status":"start","queue_duration_s":100.470743,"scheduling_latency_s":100.470795}1254gitlab # [ 225.012168] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:26.920Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"027842b6225d66517d833de8","created_at":"2026-09-13T10:12:46.189Z","trace_propagation_headers":{"sentry-trace":"dd67c4bd170847b0aa02ab8c655a70ec-5dce2b7c0486457e","baggage":"sentry-trace_id=dd67c4bd170847b0aa02ab8c655a70ec,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"87c9b4b8df264301766009d96ab47e62","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-13T10:12:46.202Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-027842b6225d66517d833de8: done: 0.247379 sec","job_status":"done","queue_duration_s":100.470743,"scheduling_latency_s":100.470795,"redis_calls":29,"redis_duration_s":0.009281000000000001,"redis_read_bytes":282,"redis_write_bytes":3632,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.005472,"redis_feature_flag_read_bytes":272,"redis_feature_flag_write_bytes":2385,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002337,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001472,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":15,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.107269,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.14541046,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.247379,"completed_at":"2026-09-13T10:14:26.920Z","load_balancing_strategy":"primary","db_duration_s":0.100255,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1255gitlab # [ 226.964256] bundle[2044]: {"timestamp":"2026-09-13T10:14:28.872Z","pid":2044,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1256gitlab # [ 226.964593] bundle[2044]: {"timestamp":"2026-09-13T10:14:28.872Z","pid":2044,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1257gitlab # [ 226.964779] bundle[2044]: {"timestamp":"2026-09-13T10:14:28.872Z","pid":2044,"message":"! #\u003cThread:0x0000fed94cfd1360 /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1258gitlab # [ 226.964946] bundle[2044]: {"timestamp":"2026-09-13T10:14:28.872Z","pid":2044,"message":"! #\u003cThread:0x0000fed94d223ac8 /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1259gitlab # [ 226.965074] bundle[2044]: {"timestamp":"2026-09-13T10:14:28.872Z","pid":2044,"message":"Use Ctrl-C to stop"}1260gitlab # [ 226.965195] bundle[2044]: {"timestamp":"2026-09-13T10:14:28.872Z","pid":2044,"message":"Sending status to systemd every 1.0 sec"}1261gitlab # [ 228.036563] bundle[2044]: {"timestamp":"2026-09-13T10:14:29.943Z","pid":2044,"message":"- Worker 0 (PID: 2969) booted in 1.03s, phase: 0"}1262gitlab # [ 228.039759] bundle[2044]: {"timestamp":"2026-09-13T10:14:29.948Z","pid":2044,"message":"- Worker 1 (PID: 2973) booted in 1.01s, phase: 0"}1263gitlab # [ 228.045857] systemd[1]: Started gitlab.service.1264gitlab # [ 228.048117] systemd[1]: Reached target Common target for all GitLab services..1265gitlab # [ 228.051297] systemd[1]: Starting setup-gitlab.service...1266gitlab # [ 228.087505] setup-gitlab-start[3006]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1267gitlab # [ 231.488955] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:14:33 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 33561268gitlab # [ 231.494448] setup-gitlab-start[3006]: + TOKEN=nixbot-test-token-of-20-chars1269gitlab # [ 231.495546] setup-gitlab-start[3006]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n'1270gitlab # [ 231.514571] runuser[3033]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1271gitlab # [ 232.082707] setup-gitlab-start[3034]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1272gitlab # [ 232.089453] setup-gitlab-start[3034]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1273gitlab # [ 232.095675] setup-gitlab-start[3034]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1274gitlab # [ 232.905607] setup-gitlab-start[3034]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1275gitlab # [ 232.905944] setup-gitlab-start[3034]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1276gitlab # [ 232.908255] setup-gitlab-start[3034]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1277gitlab # [ 236.186974] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:38.094Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"07618d129c865b30149693e9","created_at":"2026-09-13T10:14:37.995Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bd8153d6badd3dccd52adda8537cf494","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.090Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-07618d129c865b30149693e9: start","job_status":"start","queue_duration_s":0.003262,"scheduling_latency_s":0.00332}1278gitlab # [ 236.329690] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:38.238Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1279gitlab # [ 236.377167] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:38.285Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"07618d129c865b30149693e9","created_at":"2026-09-13T10:14:37.995Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bd8153d6badd3dccd52adda8537cf494","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.090Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-07618d129c865b30149693e9: done: 0.190958 sec","job_status":"done","queue_duration_s":0.003262,"scheduling_latency_s":0.00332,"redis_calls":10,"redis_duration_s":0.01772,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003486,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.008861,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.005373,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03966,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0018427,"duration_s":0.190958,"completed_at":"2026-09-13T10:14:38.284Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.018685,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1280gitlab # [ 236.466703] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:38.298Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"9373dcc60252ca64b1fb729f","created_at":"2026-09-13T10:14:38.103Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b688df25d497c8badae32c5bf6022ecc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-13T10:14:38.118Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-9373dcc60252ca64b1fb729f: start","job_status":"start","queue_duration_s":0.179412,"scheduling_latency_s":0.179941}1281gitlab # [ 236.557666] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:38.465Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"9373dcc60252ca64b1fb729f","created_at":"2026-09-13T10:14:38.103Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b688df25d497c8badae32c5bf6022ecc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-13T10:14:38.118Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-9373dcc60252ca64b1fb729f: done: 0.168014 sec","job_status":"done","queue_duration_s":0.179412,"scheduling_latency_s":0.179941,"redis_calls":10,"redis_duration_s":0.019302,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003946,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.012077,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003279,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.059179,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00155046,"duration_s":0.168014,"completed_at":"2026-09-13T10:14:38.465Z","load_balancing_strategy":"primary","db_duration_s":0.024224,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1282gitlab # [ 236.649438] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:38.491Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"b67d753f62fe07f9ed6eb3e2","created_at":"2026-09-13T10:14:38.132Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b9f621100a19b4fae72942c6374127ee","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.155Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-b67d753f62fe07f9ed6eb3e2: start","job_status":"start","queue_duration_s":0.336231,"scheduling_latency_s":0.336276}1283gitlab # [ 236.722220] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:38.630Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1284gitlab # [ 236.759438] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:38.667Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"b67d753f62fe07f9ed6eb3e2","created_at":"2026-09-13T10:14:38.132Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b9f621100a19b4fae72942c6374127ee","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.155Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-b67d753f62fe07f9ed6eb3e2: done: 0.175693 sec","job_status":"done","queue_duration_s":0.336231,"scheduling_latency_s":0.336276,"redis_calls":10,"redis_duration_s":0.019337999999999998,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.009123,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.008834,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001381,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032081,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0050457,"duration_s":0.175693,"completed_at":"2026-09-13T10:14:38.667Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002773,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1285gitlab # [ 236.851456] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:38.676Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"713353be5731388223b40fd4","created_at":"2026-09-13T10:14:38.195Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9df404e1b9e4c3df2473aa36baa7fe2f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-13T10:14:38.207Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-713353be5731388223b40fd4: start","job_status":"start","queue_duration_s":0.46918,"scheduling_latency_s":0.469223}1286gitlab # [ 236.871973] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:38.713Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1287gitlab # [ 236.876588] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:38.727Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"713353be5731388223b40fd4","created_at":"2026-09-13T10:14:38.195Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9df404e1b9e4c3df2473aa36baa7fe2f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-13T10:14:38.207Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-713353be5731388223b40fd4: done: 0.051481 sec","job_status":"done","queue_duration_s":0.46918,"scheduling_latency_s":0.469223,"redis_calls":10,"redis_duration_s":0.002379,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000499,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001358,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000522,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02031,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00036658,"duration_s":0.051481,"completed_at":"2026-09-13T10:14:38.727Z","load_balancing_strategy":"primary","db_duration_s":0.01378,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1288gitlab # [ 236.921512] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:38.729Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"1dc761f0e8f1d27c8e5363ea","created_at":"2026-09-13T10:14:38.221Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5c891e8d3c6e8778ce3bbfa1a8c612a4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.247Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-1dc761f0e8f1d27c8e5363ea: start","job_status":"start","queue_duration_s":0.482352,"scheduling_latency_s":0.482384}1289gitlab # [ 236.936272] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:38.771Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"1dc761f0e8f1d27c8e5363ea","created_at":"2026-09-13T10:14:38.221Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5c891e8d3c6e8778ce3bbfa1a8c612a4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.247Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-1dc761f0e8f1d27c8e5363ea: done: 0.041722 sec","job_status":"done","queue_duration_s":0.482352,"scheduling_latency_s":0.482384,"redis_calls":12,"redis_duration_s":0.002092,"redis_read_bytes":11,"redis_write_bytes":1482,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000651,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001124,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000317,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015834,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026264,"duration_s":0.041722,"completed_at":"2026-09-13T10:14:38.771Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008585,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1290gitlab # [ 236.977661] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:38.772Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"84a9a9403a2ce0e0217b6073","created_at":"2026-09-13T10:14:38.277Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.294Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-84a9a9403a2ce0e0217b6073: start","job_status":"start","queue_duration_s":0.4784,"scheduling_latency_s":0.47842}1291gitlab # [ 237.222212] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:39.129Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"84a9a9403a2ce0e0217b6073","created_at":"2026-09-13T10:14:38.277Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.294Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-84a9a9403a2ce0e0217b6073: done: 0.35661 sec","job_status":"done","queue_duration_s":0.4784,"scheduling_latency_s":0.47842,"redis_calls":71,"redis_duration_s":0.045224,"redis_read_bytes":120,"redis_write_bytes":42624,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.007591,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.001499,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.028824,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00731,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.052,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.052,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.170504,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00039874,"duration_s":0.35661,"completed_at":"2026-09-13T10:14:39.129Z","load_balancing_strategy":"primary","db_duration_s":0.051725,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1292gitlab # [ 237.318031] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:39.131Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"7a85b48d414927c669e45124","created_at":"2026-09-13T10:14:38.354Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"098c7b26431a0f611f4b224471d4c87c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.376Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-7a85b48d414927c669e45124: start","job_status":"start","queue_duration_s":0.75482,"scheduling_latency_s":0.754847}1293gitlab # [ 237.334825] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:39.180Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"7a85b48d414927c669e45124","created_at":"2026-09-13T10:14:38.354Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"098c7b26431a0f611f4b224471d4c87c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.376Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-7a85b48d414927c669e45124: done: 0.049205 sec","job_status":"done","queue_duration_s":0.75482,"scheduling_latency_s":0.754847,"redis_calls":19,"redis_duration_s":0.005224,"redis_read_bytes":22,"redis_write_bytes":4785,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001418,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000379,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.002575,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000852,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033709,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00036014,"duration_s":0.049205,"completed_at":"2026-09-13T10:14:39.180Z","load_balancing_strategy":"primary","db_duration_s":0.004639,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1294gitlab # [ 237.378649] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:39.182Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"4381418025462c0d914b43b2","created_at":"2026-09-13T10:14:38.407Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"99e84ca5ab087ca9d22ef89e7035e700","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.429Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-4381418025462c0d914b43b2: start","job_status":"start","queue_duration_s":0.753563,"scheduling_latency_s":0.753587}1295gitlab # [ 237.662441] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:39.570Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"4381418025462c0d914b43b2","created_at":"2026-09-13T10:14:38.407Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"99e84ca5ab087ca9d22ef89e7035e700","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.429Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-4381418025462c0d914b43b2: done: 0.38827 sec","job_status":"done","queue_duration_s":0.753563,"scheduling_latency_s":0.753587,"redis_calls":565,"redis_duration_s":0.08725400000000001,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000514,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001268,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.085472,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.338969,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003088,"duration_s":0.38827,"completed_at":"2026-09-13T10:14:39.570Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000813,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1296gitlab # [ 237.723915] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:39.572Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"5ce08d3bccf17573a3f6d95f","created_at":"2026-09-13T10:14:38.455Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"017ce7a4b7ca06acfaa93fc07c859d4b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-13T10:14:38.472Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-5ce08d3bccf17573a3f6d95f: start","job_status":"start","queue_duration_s":1.099926,"scheduling_latency_s":1.099956}1297gitlab # [ 237.738177] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:39.596Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1298gitlab # [ 237.741939] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:39.616Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"5ce08d3bccf17573a3f6d95f","created_at":"2026-09-13T10:14:38.455Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"017ce7a4b7ca06acfaa93fc07c859d4b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-13T10:14:38.472Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-5ce08d3bccf17573a3f6d95f: done: 0.04367 sec","job_status":"done","queue_duration_s":1.099926,"scheduling_latency_s":1.099956,"redis_calls":10,"redis_duration_s":0.003394,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000574,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002091,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000729,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028371,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0005584,"duration_s":0.04367,"completed_at":"2026-09-13T10:14:39.616Z","load_balancing_strategy":"primary","db_duration_s":0.000867,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1299gitlab # [ 237.782732] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:39.619Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"5a01f758e6c94de319a3bb95","created_at":"2026-09-13T10:14:38.489Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a5ac1e76768f667d0dd51435b048dcce","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.504Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-5a01f758e6c94de319a3bb95: start","job_status":"start","queue_duration_s":1.115466,"scheduling_latency_s":1.115515}1300gitlab # [ 237.860166] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:39.767Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"5a01f758e6c94de319a3bb95","created_at":"2026-09-13T10:14:38.489Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a5ac1e76768f667d0dd51435b048dcce","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.504Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-5a01f758e6c94de319a3bb95: done: 0.147629 sec","job_status":"done","queue_duration_s":1.115466,"scheduling_latency_s":1.115515,"redis_calls":14,"redis_duration_s":0.005074,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001244,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002253,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001577,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.02,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.106926,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00138204,"exclusive_lock_hold_duration_s":0.09203614,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.147629,"completed_at":"2026-09-13T10:14:39.766Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.023823,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1301gitlab # [ 237.938940] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:39.771Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"198d9c4c70625661084bd763","created_at":"2026-09-13T10:14:38.516Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5996d2d2fdf568276e60f86557357783","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.526Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-198d9c4c70625661084bd763: start","job_status":"start","queue_duration_s":1.245351,"scheduling_latency_s":1.245391}1302gitlab # [ 237.955241] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:39.799Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1303gitlab # [ 237.958715] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:39.814Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"198d9c4c70625661084bd763","created_at":"2026-09-13T10:14:38.516Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5996d2d2fdf568276e60f86557357783","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.526Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-198d9c4c70625661084bd763: done: 0.043355 sec","job_status":"done","queue_duration_s":1.245351,"scheduling_latency_s":1.245391,"redis_calls":10,"redis_duration_s":0.003788,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000924,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002102,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000762,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025632,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00060316,"duration_s":0.043355,"completed_at":"2026-09-13T10:14:39.814Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00107,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1304gitlab # [ 238.000841] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:39.817Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"fe5680450cf5c1ae1ffa8239","created_at":"2026-09-13T10:14:38.565Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a088c511b443a28417fc5badbee0a6f5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.574Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-fe5680450cf5c1ae1ffa8239: start","job_status":"start","queue_duration_s":1.242547,"scheduling_latency_s":1.242582}1305gitlab # [ 238.015437] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:39.853Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"fe5680450cf5c1ae1ffa8239","created_at":"2026-09-13T10:14:38.565Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a088c511b443a28417fc5badbee0a6f5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.574Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-fe5680450cf5c1ae1ffa8239: done: 0.03575 sec","job_status":"done","queue_duration_s":1.242547,"scheduling_latency_s":1.242582,"redis_calls":10,"redis_duration_s":0.0033090000000000003,"redis_read_bytes":9,"redis_write_bytes":1299,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00078,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001845,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000684,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021108,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00050748,"duration_s":0.03575,"completed_at":"2026-09-13T10:14:39.853Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000691,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1306gitlab # [ 238.059090] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:39.855Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"f5b5a1e4cb37c95c58413cb0","created_at":"2026-09-13T10:14:38.583Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"47e180db45cb5409bdbb96f775ebf103","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.593Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-f5b5a1e4cb37c95c58413cb0: start","job_status":"start","queue_duration_s":1.262554,"scheduling_latency_s":1.262583}1307gitlab # [ 238.073566] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:39.947Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"f5b5a1e4cb37c95c58413cb0","created_at":"2026-09-13T10:14:38.583Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"47e180db45cb5409bdbb96f775ebf103","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.593Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-f5b5a1e4cb37c95c58413cb0: done: 0.091234 sec","job_status":"done","queue_duration_s":1.262554,"scheduling_latency_s":1.262583,"redis_calls":16,"redis_duration_s":0.00577,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.003283,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001846,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000641,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.052815,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00063934,"duration_s":0.091234,"completed_at":"2026-09-13T10:14:39.946Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.020743,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1308gitlab # [ 238.115081] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:39.950Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"9d22262a3332f5a66535b8ec","created_at":"2026-09-13T10:14:38.621Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d037c1ff93be3231a5339bda2e4332f4","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.644Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-9d22262a3332f5a66535b8ec: start","job_status":"start","queue_duration_s":1.305746,"scheduling_latency_s":1.305781}1309gitlab # [ 238.128676] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:40.002Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"9d22262a3332f5a66535b8ec","created_at":"2026-09-13T10:14:38.621Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d037c1ff93be3231a5339bda2e4332f4","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.644Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-9d22262a3332f5a66535b8ec: done: 0.052096 sec","job_status":"done","queue_duration_s":1.305746,"scheduling_latency_s":1.305781,"redis_calls":12,"redis_duration_s":0.003416,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000481,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001631,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001304,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031361,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00079302,"exclusive_lock_hold_duration_s":0.01433484,"duration_s":0.052096,"completed_at":"2026-09-13T10:14:40.002Z","load_balancing_strategy":"primary","db_duration_s":0.006378,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1310gitlab # [ 238.169734] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:40.004Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"548d2f087d668fe9a8119fe0","created_at":"2026-09-13T10:14:38.664Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4a8f5181cf490ae6c351642f210bd417","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.699Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-548d2f087d668fe9a8119fe0: start","job_status":"start","queue_duration_s":1.30525,"scheduling_latency_s":1.305279}1311gitlab # [ 238.447289] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:40.354Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"548d2f087d668fe9a8119fe0","created_at":"2026-09-13T10:14:38.664Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4a8f5181cf490ae6c351642f210bd417","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.699Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-548d2f087d668fe9a8119fe0: done: 0.350159 sec","job_status":"done","queue_duration_s":1.30525,"scheduling_latency_s":1.305279,"redis_calls":14,"redis_duration_s":0.003845,"redis_read_bytes":13,"redis_write_bytes":1645,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000903,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001651,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001291,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.057,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.057,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.268027,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00075424,"exclusive_lock_hold_duration_s":0.30653423999999996,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.350159,"completed_at":"2026-09-13T10:14:40.354Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.087026,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1312gitlab # [ 238.508265] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:40.359Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"bfae73f6edf3b8133de8e4ed","created_at":"2026-09-13T10:14:38.709Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4aa6c5186ccbb22e17df217629fb38e2","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.732Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-bfae73f6edf3b8133de8e4ed: start","job_status":"start","queue_duration_s":1.6266,"scheduling_latency_s":1.626651}1313gitlab # [ 238.522421] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:40.409Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"bfae73f6edf3b8133de8e4ed","created_at":"2026-09-13T10:14:38.709Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4aa6c5186ccbb22e17df217629fb38e2","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.732Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-bfae73f6edf3b8133de8e4ed: done: 0.050202 sec","job_status":"done","queue_duration_s":1.6266,"scheduling_latency_s":1.626651,"redis_calls":12,"redis_duration_s":0.004801,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000992,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000867,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002186,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000756,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030512,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00058022,"duration_s":0.050202,"completed_at":"2026-09-13T10:14:40.409Z","load_balancing_strategy":"primary","db_duration_s":0.00088,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1314gitlab # [ 238.564226] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:40.411Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"5b57732be978d2d844077156","created_at":"2026-09-13T10:14:38.736Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"92f004ab2429fb8569ba98fd0fcf8931","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.755Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-5b57732be978d2d844077156: start","job_status":"start","queue_duration_s":1.655856,"scheduling_latency_s":1.655885}1315gitlab # [ 238.745515] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:40.654Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"5b57732be978d2d844077156","created_at":"2026-09-13T10:14:38.736Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"92f004ab2429fb8569ba98fd0fcf8931","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.755Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-5b57732be978d2d844077156: done: 0.242321 sec","job_status":"done","queue_duration_s":1.655856,"scheduling_latency_s":1.655885,"redis_calls":12,"redis_duration_s":0.003927,"redis_read_bytes":11,"redis_write_bytes":1447,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000602,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":319,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001835,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":586,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00149,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":542,"db_count":101,"db_write_count":0,"db_cached_count":99,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":101,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":99,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":100,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":99,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.207495,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00076994,"exclusive_lock_hold_duration_s":0.21310656,"extra.ci_update_locked_unknown_artifacts_worker.removed_count":0,"extra.ci_update_locked_unknown_artifacts_worker.locked_count":0,"duration_s":0.242321,"completed_at":"2026-09-13T10:14:40.654Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.026937,"urgency":"throttled","target_duration_s":300,"target_scheduling_latency_s":null}1316gitlab # [ 238.842073] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:40.656Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"697b0fc1768f610a16352be3","created_at":"2026-09-13T10:14:38.759Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5ef85e359f112c53d766ee4e96b6bfd5","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.774Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-697b0fc1768f610a16352be3: start","job_status":"start","queue_duration_s":1.881436,"scheduling_latency_s":1.881456}1317gitlab # [ 238.859379] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:40.685Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"697b0fc1768f610a16352be3","created_at":"2026-09-13T10:14:38.759Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5ef85e359f112c53d766ee4e96b6bfd5","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.774Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-697b0fc1768f610a16352be3: done: 0.029293 sec","job_status":"done","queue_duration_s":1.881436,"scheduling_latency_s":1.881456,"redis_calls":10,"redis_duration_s":0.002843,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000589,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001652,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000602,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015753,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0004089,"duration_s":0.029293,"completed_at":"2026-09-13T10:14:40.685Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004466,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1318gitlab # [ 238.903943] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:40.687Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"a064cc303aea6200726cb972","created_at":"2026-09-13T10:14:38.790Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c7529e437503d21fc6f0273e007a631e","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.806Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-a064cc303aea6200726cb972: start","job_status":"start","queue_duration_s":1.881076,"scheduling_latency_s":1.881093}1319gitlab # [ 238.918454] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:40.738Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"a064cc303aea6200726cb972","created_at":"2026-09-13T10:14:38.790Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c7529e437503d21fc6f0273e007a631e","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.806Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-a064cc303aea6200726cb972: done: 0.050908 sec","job_status":"done","queue_duration_s":1.881076,"scheduling_latency_s":1.881093,"redis_calls":10,"redis_duration_s":0.002986,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000521,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001825,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00064,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01631,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0005167,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.050908,"completed_at":"2026-09-13T10:14:40.738Z","load_balancing_strategy":"primary","db_duration_s":0.025079,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1320gitlab # [ 238.961505] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:40.740Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"c2b95a4dbd1ac19d84c86898","created_at":"2026-09-13T10:14:38.810Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"539049663750ab7838c5b7e59fb96ff8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.822Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-c2b95a4dbd1ac19d84c86898: start","job_status":"start","queue_duration_s":1.91774,"scheduling_latency_s":1.91776}1321gitlab # [ 238.975867] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:40.798Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"c2b95a4dbd1ac19d84c86898","created_at":"2026-09-13T10:14:38.810Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"539049663750ab7838c5b7e59fb96ff8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.822Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-c2b95a4dbd1ac19d84c86898: done: 0.058148 sec","job_status":"done","queue_duration_s":1.91774,"scheduling_latency_s":1.91776,"redis_calls":12,"redis_duration_s":0.003708,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000597,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001588,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001523,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025321,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00093386,"exclusive_lock_hold_duration_s":0.02239408,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.058148,"completed_at":"2026-09-13T10:14:40.798Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.022575,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1322gitlab # [ 239.019980] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:40.800Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"46694d4b3cffa948ff6bbce8","created_at":"2026-09-13T10:14:38.825Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6783461f8acba08c62ba2bf3021530b3","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.837Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-46694d4b3cffa948ff6bbce8: start","job_status":"start","queue_duration_s":1.962712,"scheduling_latency_s":1.962733}1323gitlab # [ 239.034311] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:40.829Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"46694d4b3cffa948ff6bbce8","created_at":"2026-09-13T10:14:38.825Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6783461f8acba08c62ba2bf3021530b3","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.837Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-46694d4b3cffa948ff6bbce8: done: 0.028833 sec","job_status":"done","queue_duration_s":1.962712,"scheduling_latency_s":1.962733,"redis_calls":12,"redis_duration_s":0.003339,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000564,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001524,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001251,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016115,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00064728,"exclusive_lock_hold_duration_s":0.0044328399999999995,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.028833,"completed_at":"2026-09-13T10:14:40.828Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002778,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1324gitlab # [ 239.077030] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:40.830Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"8254b474e79462ae13d833ee","created_at":"2026-09-13T10:14:38.863Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"345b4e6e662760633466e81473fe1b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.877Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-8254b474e79462ae13d833ee: start","job_status":"start","queue_duration_s":1.953135,"scheduling_latency_s":1.953153}1325gitlab # [ 239.092593] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:40.866Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"8254b474e79462ae13d833ee","created_at":"2026-09-13T10:14:38.863Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"345b4e6e662760633466e81473fe1b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.877Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-8254b474e79462ae13d833ee: done: 0.035894 sec","job_status":"done","queue_duration_s":1.953135,"scheduling_latency_s":1.953153,"redis_calls":13,"redis_duration_s":0.003982,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000502,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002259,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001221,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022383,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01099,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.035894,"completed_at":"2026-09-13T10:14:40.866Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003997,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1326gitlab # [ 239.136669] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:40.868Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"9c8e5742a622f7013b8d7d19","created_at":"2026-09-13T10:14:38.893Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"966d0aab48a70932d6af53aaea3ae0ea","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.911Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-9c8e5742a622f7013b8d7d19: start","job_status":"start","queue_duration_s":1.956574,"scheduling_latency_s":1.956592}1327gitlab # [ 239.150668] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:40.899Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"9c8e5742a622f7013b8d7d19","created_at":"2026-09-13T10:14:38.893Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"966d0aab48a70932d6af53aaea3ae0ea","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.911Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-9c8e5742a622f7013b8d7d19: done: 0.030902 sec","job_status":"done","queue_duration_s":1.956574,"scheduling_latency_s":1.956592,"redis_calls":10,"redis_duration_s":0.002778,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000535,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001685,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000558,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016845,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00048234,"duration_s":0.030902,"completed_at":"2026-09-13T10:14:40.899Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006048,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1328gitlab # [ 239.192354] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:40.900Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"baca2219eefc112b6a53a14d","created_at":"2026-09-13T10:14:38.914Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"64edb642f121f05e0d198d9e2f9e4a5f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.931Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-baca2219eefc112b6a53a14d: start","job_status":"start","queue_duration_s":1.968616,"scheduling_latency_s":1.968632}1329gitlab # [ 239.206576] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:40.912Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1330gitlab # [ 239.210221] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:40.920Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"baca2219eefc112b6a53a14d","created_at":"2026-09-13T10:14:38.914Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"64edb642f121f05e0d198d9e2f9e4a5f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.931Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-baca2219eefc112b6a53a14d: done: 0.02014 sec","job_status":"done","queue_duration_s":1.968616,"scheduling_latency_s":1.968632,"redis_calls":10,"redis_duration_s":0.002366,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000528,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001367,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000471,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011081,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003227,"duration_s":0.02014,"completed_at":"2026-09-13T10:14:40.920Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000637,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1331gitlab # [ 239.251693] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:40.922Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"6c3914f48efa954ad2d883a7","created_at":"2026-09-13T10:14:38.936Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c116f73a32cd28041e676c631f981481","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.960Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-6c3914f48efa954ad2d883a7: start","job_status":"start","queue_duration_s":1.961121,"scheduling_latency_s":1.961138}1332gitlab # [ 239.266007] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:40.954Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"6c3914f48efa954ad2d883a7","created_at":"2026-09-13T10:14:38.936Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c116f73a32cd28041e676c631f981481","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-13T10:14:38.960Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-6c3914f48efa954ad2d883a7: done: 0.031969 sec","job_status":"done","queue_duration_s":1.961121,"scheduling_latency_s":1.961138,"redis_calls":12,"redis_duration_s":0.008083,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000693,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.006243,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001147,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014933,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00059774,"exclusive_lock_hold_duration_s":0.011299739999999999,"duration_s":0.031969,"completed_at":"2026-09-13T10:14:40.953Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00404,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1333gitlab # [ 239.308745] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:40.956Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d1da38986e1978f2a8d88d8","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d1da38986e1978f2a8d88d8: start","job_status":"start","queue_duration_s":1.840014,"scheduling_latency_s":1.840033}1334gitlab # [ 239.320698] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:40.979Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d1da38986e1978f2a8d88d8","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d1da38986e1978f2a8d88d8: done: 0.023736 sec","job_status":"done","queue_duration_s":1.840014,"scheduling_latency_s":1.840033,"redis_calls":16,"redis_duration_s":0.0027559999999999998,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000476,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000668,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001612,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014555,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00023474,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023736,"completed_at":"2026-09-13T10:14:40.979Z","load_balancing_strategy":"primary","db_duration_s":0.000742,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1335gitlab # [ 239.361183] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:40.981Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1329d26fb73c9738bc42122e","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1329d26fb73c9738bc42122e: start","job_status":"start","queue_duration_s":1.865338,"scheduling_latency_s":1.865356}1336gitlab # [ 239.373183] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:40.997Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1329d26fb73c9738bc42122e","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1329d26fb73c9738bc42122e: done: 0.015553 sec","job_status":"done","queue_duration_s":1.865338,"scheduling_latency_s":1.865356,"redis_calls":13,"redis_duration_s":0.00162,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000601,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001019,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01059,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015553,"completed_at":"2026-09-13T10:14:40.997Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1337gitlab # [ 239.411709] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:40.998Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2eb19719e2595544cec59ec9","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2eb19719e2595544cec59ec9: start","job_status":"start","queue_duration_s":1.882165,"scheduling_latency_s":1.882182}1338gitlab # [ 239.424263] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.013Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2eb19719e2595544cec59ec9","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2eb19719e2595544cec59ec9: done: 0.014892 sec","job_status":"done","queue_duration_s":1.882165,"scheduling_latency_s":1.882182,"redis_calls":13,"redis_duration_s":0.0015249999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000542,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000983,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009957,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014892,"completed_at":"2026-09-13T10:14:41.013Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1339gitlab # [ 239.463586] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.014Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"260c666d878113f77b816c89","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-260c666d878113f77b816c89: start","job_status":"start","queue_duration_s":1.898203,"scheduling_latency_s":1.89822}1340gitlab # [ 239.476164] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.028Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"260c666d878113f77b816c89","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-260c666d878113f77b816c89: done: 0.014409 sec","job_status":"done","queue_duration_s":1.898203,"scheduling_latency_s":1.89822,"redis_calls":13,"redis_duration_s":0.001521,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000982,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009982,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014409,"completed_at":"2026-09-13T10:14:41.028Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1341gitlab # [ 239.515877] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.030Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c6699dbe0e89f58f6540a113","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c6699dbe0e89f58f6540a113: start","job_status":"start","queue_duration_s":1.913854,"scheduling_latency_s":1.91387}1342gitlab # [ 239.528332] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.044Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c6699dbe0e89f58f6540a113","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c6699dbe0e89f58f6540a113: done: 0.014631 sec","job_status":"done","queue_duration_s":1.913854,"scheduling_latency_s":1.91387,"redis_calls":13,"redis_duration_s":0.001486,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000543,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000943,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010314,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014631,"completed_at":"2026-09-13T10:14:41.044Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1343gitlab # [ 239.567047] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.045Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6322a1003e72ddc37a9c1ed9","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6322a1003e72ddc37a9c1ed9: start","job_status":"start","queue_duration_s":1.929774,"scheduling_latency_s":1.92979}1344gitlab # [ 239.579462] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.060Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6322a1003e72ddc37a9c1ed9","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6322a1003e72ddc37a9c1ed9: done: 0.014619 sec","job_status":"done","queue_duration_s":1.929774,"scheduling_latency_s":1.92979,"redis_calls":13,"redis_duration_s":0.001617,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000684,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000933,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010347,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014619,"completed_at":"2026-09-13T10:14:41.060Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1345gitlab # [ 239.618311] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.061Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"433b872a2d406d28a825f769","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-433b872a2d406d28a825f769: start","job_status":"start","queue_duration_s":1.945525,"scheduling_latency_s":1.945541}1346gitlab # [ 239.630740] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.077Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"433b872a2d406d28a825f769","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-433b872a2d406d28a825f769: done: 0.015478 sec","job_status":"done","queue_duration_s":1.945525,"scheduling_latency_s":1.945541,"redis_calls":13,"redis_duration_s":0.001521,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000985,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010435,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015478,"completed_at":"2026-09-13T10:14:41.077Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1347gitlab # [ 239.669381] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.078Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"edaf88561327445327df04d1","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-edaf88561327445327df04d1: start","job_status":"start","queue_duration_s":1.962255,"scheduling_latency_s":1.962271}1348gitlab # [ 239.681695] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.094Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"edaf88561327445327df04d1","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-edaf88561327445327df04d1: done: 0.01631 sec","job_status":"done","queue_duration_s":1.962255,"scheduling_latency_s":1.962271,"redis_calls":13,"redis_duration_s":0.001703,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000573,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00113,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010577,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01631,"completed_at":"2026-09-13T10:14:41.094Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1349gitlab # [ 239.721963] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.095Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"944b9e4af8df095e7eb8bf50","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-944b9e4af8df095e7eb8bf50: start","job_status":"start","queue_duration_s":1.979617,"scheduling_latency_s":1.979633}1350gitlab # [ 239.734326] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.110Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"944b9e4af8df095e7eb8bf50","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-944b9e4af8df095e7eb8bf50: done: 0.015084 sec","job_status":"done","queue_duration_s":1.979617,"scheduling_latency_s":1.979633,"redis_calls":13,"redis_duration_s":0.001568,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00058,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000988,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010287,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015084,"completed_at":"2026-09-13T10:14:41.110Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1351gitlab # [ 239.773064] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.111Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4e35c1f3f4f5113d6162496d","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4e35c1f3f4f5113d6162496d: start","job_status":"start","queue_duration_s":1.995739,"scheduling_latency_s":1.995755}1352gitlab # [ 239.785263] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.126Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4e35c1f3f4f5113d6162496d","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4e35c1f3f4f5113d6162496d: done: 0.014831 sec","job_status":"done","queue_duration_s":1.995739,"scheduling_latency_s":1.995755,"redis_calls":13,"redis_duration_s":0.001522,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000535,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000987,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010137,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014831,"completed_at":"2026-09-13T10:14:41.126Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1353gitlab # [ 239.823754] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.127Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e5a672c4481d04c10dd0b03d","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e5a672c4481d04c10dd0b03d: start","job_status":"start","queue_duration_s":2.011783,"scheduling_latency_s":2.011799}1354gitlab # [ 239.836192] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.142Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e5a672c4481d04c10dd0b03d","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e5a672c4481d04c10dd0b03d: done: 0.014562 sec","job_status":"done","queue_duration_s":2.011783,"scheduling_latency_s":2.011799,"redis_calls":13,"redis_duration_s":0.001482,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000541,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000941,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010081,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014562,"completed_at":"2026-09-13T10:14:41.142Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1355gitlab # [ 239.874537] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.143Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a62a252ced753d011c5b4319","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a62a252ced753d011c5b4319: start","job_status":"start","queue_duration_s":2.027607,"scheduling_latency_s":2.027623}1356gitlab # [ 239.886915] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.269Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a62a252ced753d011c5b4319","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a62a252ced753d011c5b4319: done: 0.126114 sec","job_status":"done","queue_duration_s":2.027607,"scheduling_latency_s":2.027623,"redis_calls":13,"redis_duration_s":0.001994,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000868,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001126,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.120468,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.126114,"completed_at":"2026-09-13T10:14:41.269Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1357gitlab # [ 239.926577] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.271Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2cf6f8e22cde5adbf9709616","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2cf6f8e22cde5adbf9709616: start","job_status":"start","queue_duration_s":2.154872,"scheduling_latency_s":2.154891}1358gitlab # [ 239.938775] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.285Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2cf6f8e22cde5adbf9709616","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2cf6f8e22cde5adbf9709616: done: 0.014909 sec","job_status":"done","queue_duration_s":2.154872,"scheduling_latency_s":2.154891,"redis_calls":13,"redis_duration_s":0.001541,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000562,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000979,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010469,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014909,"completed_at":"2026-09-13T10:14:41.285Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1359gitlab # [ 239.977837] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.287Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cfe6d4a0c850d3855a17ffe9","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cfe6d4a0c850d3855a17ffe9: start","job_status":"start","queue_duration_s":2.17083,"scheduling_latency_s":2.170846}1360gitlab # [ 239.989894] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.300Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cfe6d4a0c850d3855a17ffe9","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cfe6d4a0c850d3855a17ffe9: done: 0.013765 sec","job_status":"done","queue_duration_s":2.17083,"scheduling_latency_s":2.170846,"redis_calls":13,"redis_duration_s":0.001513,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000581,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000932,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009441,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013765,"completed_at":"2026-09-13T10:14:41.300Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1361gitlab # [ 240.028516] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.302Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9338e12646af00e0eb5e4e71","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9338e12646af00e0eb5e4e71: start","job_status":"start","queue_duration_s":2.186282,"scheduling_latency_s":2.186298}1362gitlab # [ 240.040378] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.317Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9338e12646af00e0eb5e4e71","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9338e12646af00e0eb5e4e71: done: 0.014722 sec","job_status":"done","queue_duration_s":2.186282,"scheduling_latency_s":2.186298,"redis_calls":13,"redis_duration_s":0.0014910000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000959,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01033,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014722,"completed_at":"2026-09-13T10:14:41.317Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1363gitlab # [ 240.079344] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.318Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"588c55360e02a0d2192f49b6","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-588c55360e02a0d2192f49b6: start","job_status":"start","queue_duration_s":2.202093,"scheduling_latency_s":2.202109}1364gitlab # [ 240.091841] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.333Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"588c55360e02a0d2192f49b6","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-588c55360e02a0d2192f49b6: done: 0.014767 sec","job_status":"done","queue_duration_s":2.202093,"scheduling_latency_s":2.202109,"redis_calls":13,"redis_duration_s":0.001527,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000551,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000976,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010445,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014767,"completed_at":"2026-09-13T10:14:41.332Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1365gitlab # [ 240.132030] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.334Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"01d07ddcbd368ba4c5c901c4","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-01d07ddcbd368ba4c5c901c4: start","job_status":"start","queue_duration_s":2.217924,"scheduling_latency_s":2.21794}1366gitlab # [ 240.144573] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.348Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"01d07ddcbd368ba4c5c901c4","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-01d07ddcbd368ba4c5c901c4: done: 0.014265 sec","job_status":"done","queue_duration_s":2.217924,"scheduling_latency_s":2.21794,"redis_calls":13,"redis_duration_s":0.00145,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000927,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010068,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014265,"completed_at":"2026-09-13T10:14:41.348Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1367gitlab # [ 240.183990] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.349Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e95f8e7f27d224ead4e9029","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e95f8e7f27d224ead4e9029: start","job_status":"start","queue_duration_s":2.233288,"scheduling_latency_s":2.233303}1368gitlab # [ 240.196219] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.364Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e95f8e7f27d224ead4e9029","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e95f8e7f27d224ead4e9029: done: 0.015286 sec","job_status":"done","queue_duration_s":2.233288,"scheduling_latency_s":2.233303,"redis_calls":13,"redis_duration_s":0.00146,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000541,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000919,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009497,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015286,"completed_at":"2026-09-13T10:14:41.364Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1369gitlab # [ 240.234431] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.366Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"22677cb650973359a11296d6","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-22677cb650973359a11296d6: start","job_status":"start","queue_duration_s":2.250017,"scheduling_latency_s":2.250033}1370gitlab # [ 240.246585] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.380Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"22677cb650973359a11296d6","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-22677cb650973359a11296d6: done: 0.014723 sec","job_status":"done","queue_duration_s":2.250017,"scheduling_latency_s":2.250033,"redis_calls":13,"redis_duration_s":0.0015340000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000582,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000952,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010322,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014723,"completed_at":"2026-09-13T10:14:41.380Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1371gitlab # [ 240.284947] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.382Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e2f113737ba58ff275c59425","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e2f113737ba58ff275c59425: start","job_status":"start","queue_duration_s":2.265853,"scheduling_latency_s":2.265869}1372gitlab # [ 240.297165] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.396Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e2f113737ba58ff275c59425","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e2f113737ba58ff275c59425: done: 0.014473 sec","job_status":"done","queue_duration_s":2.265853,"scheduling_latency_s":2.265869,"redis_calls":13,"redis_duration_s":0.0014659999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000522,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000944,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010244,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014473,"completed_at":"2026-09-13T10:14:41.396Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1373gitlab # [ 240.336589] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.397Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"471d5ae77c7866dceec89383","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-471d5ae77c7866dceec89383: start","job_status":"start","queue_duration_s":2.281466,"scheduling_latency_s":2.281483}1374gitlab # [ 240.348632] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.412Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"471d5ae77c7866dceec89383","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-471d5ae77c7866dceec89383: done: 0.01467 sec","job_status":"done","queue_duration_s":2.281466,"scheduling_latency_s":2.281483,"redis_calls":13,"redis_duration_s":0.0015279999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000553,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000975,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010381,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01467,"completed_at":"2026-09-13T10:14:41.412Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1375gitlab # [ 240.387369] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.413Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1b7d2136618c638f175a1004","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1b7d2136618c638f175a1004: start","job_status":"start","queue_duration_s":2.297289,"scheduling_latency_s":2.297305}1376gitlab # [ 240.399593] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.427Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1b7d2136618c638f175a1004","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1b7d2136618c638f175a1004: done: 0.014286 sec","job_status":"done","queue_duration_s":2.297289,"scheduling_latency_s":2.297305,"redis_calls":13,"redis_duration_s":0.001434,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000913,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009377,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014286,"completed_at":"2026-09-13T10:14:41.427Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1377gitlab # [ 240.438286] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.429Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ac93d2f0bf8d18e45b83d044","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ac93d2f0bf8d18e45b83d044: start","job_status":"start","queue_duration_s":2.313341,"scheduling_latency_s":2.313357}1378gitlab # [ 240.450536] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.443Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ac93d2f0bf8d18e45b83d044","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ac93d2f0bf8d18e45b83d044: done: 0.014376 sec","job_status":"done","queue_duration_s":2.313341,"scheduling_latency_s":2.313357,"redis_calls":13,"redis_duration_s":0.002149,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001623,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010172,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014376,"completed_at":"2026-09-13T10:14:41.443Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1379gitlab # [ 240.489944] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.445Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f48b02c38539980f054ad5bf","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f48b02c38539980f054ad5bf: start","job_status":"start","queue_duration_s":2.328806,"scheduling_latency_s":2.328822}1380gitlab # [ 240.502020] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.459Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f48b02c38539980f054ad5bf","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f48b02c38539980f054ad5bf: done: 0.014641 sec","job_status":"done","queue_duration_s":2.328806,"scheduling_latency_s":2.328822,"redis_calls":13,"redis_duration_s":0.0015639999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000538,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001026,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01034,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014641,"completed_at":"2026-09-13T10:14:41.459Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1381gitlab # [ 240.541353] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.460Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bab7031cf4e4831f7e9f864c","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bab7031cf4e4831f7e9f864c: start","job_status":"start","queue_duration_s":2.344586,"scheduling_latency_s":2.344602}1382gitlab # [ 240.553301] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.475Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bab7031cf4e4831f7e9f864c","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bab7031cf4e4831f7e9f864c: done: 0.014666 sec","job_status":"done","queue_duration_s":2.344586,"scheduling_latency_s":2.344602,"redis_calls":13,"redis_duration_s":0.0014789999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000542,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000937,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010252,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014666,"completed_at":"2026-09-13T10:14:41.475Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1383gitlab # [ 240.591688] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.477Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"753b07a5ef76f335bf81ca4b","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-753b07a5ef76f335bf81ca4b: start","job_status":"start","queue_duration_s":2.360982,"scheduling_latency_s":2.361016}1384gitlab # [ 240.604043] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.491Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"753b07a5ef76f335bf81ca4b","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-753b07a5ef76f335bf81ca4b: done: 0.01459 sec","job_status":"done","queue_duration_s":2.360982,"scheduling_latency_s":2.361016,"redis_calls":13,"redis_duration_s":0.001513,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000542,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000971,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009571,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01459,"completed_at":"2026-09-13T10:14:41.491Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1385gitlab # [ 240.641913] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.493Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"934867f1b010723db72702ed","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-934867f1b010723db72702ed: start","job_status":"start","queue_duration_s":2.37681,"scheduling_latency_s":2.376826}1386gitlab # [ 240.654028] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.507Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"934867f1b010723db72702ed","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-934867f1b010723db72702ed: done: 0.01473 sec","job_status":"done","queue_duration_s":2.37681,"scheduling_latency_s":2.376826,"redis_calls":13,"redis_duration_s":0.001599,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001071,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010343,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01473,"completed_at":"2026-09-13T10:14:41.507Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1387gitlab # [ 240.692223] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.508Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b757e14f83e18d3bc7ac2537","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b757e14f83e18d3bc7ac2537: start","job_status":"start","queue_duration_s":2.392608,"scheduling_latency_s":2.392625}1388gitlab # [ 240.704054] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.523Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b757e14f83e18d3bc7ac2537","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b757e14f83e18d3bc7ac2537: done: 0.014367 sec","job_status":"done","queue_duration_s":2.392608,"scheduling_latency_s":2.392625,"redis_calls":13,"redis_duration_s":0.001484,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000564,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00092,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010122,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014367,"completed_at":"2026-09-13T10:14:41.523Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1389gitlab # [ 240.742950] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.524Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3f7729971e51addb306cf9b9","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3f7729971e51addb306cf9b9: start","job_status":"start","queue_duration_s":2.408063,"scheduling_latency_s":2.408078}1390gitlab # [ 240.755263] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.538Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3f7729971e51addb306cf9b9","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3f7729971e51addb306cf9b9: done: 0.014733 sec","job_status":"done","queue_duration_s":2.408063,"scheduling_latency_s":2.408078,"redis_calls":13,"redis_duration_s":0.001588,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000543,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001045,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009619,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014733,"completed_at":"2026-09-13T10:14:41.538Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1391gitlab # [ 240.793330] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.540Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"53bfce765478d8d059a62ce5","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-53bfce765478d8d059a62ce5: start","job_status":"start","queue_duration_s":2.424411,"scheduling_latency_s":2.424428}1392gitlab # [ 240.805591] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.554Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"53bfce765478d8d059a62ce5","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-53bfce765478d8d059a62ce5: done: 0.01439 sec","job_status":"done","queue_duration_s":2.424411,"scheduling_latency_s":2.424428,"redis_calls":13,"redis_duration_s":0.001467,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00053,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000937,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010084,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01439,"completed_at":"2026-09-13T10:14:41.554Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1393gitlab # [ 240.843817] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.556Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc42edd949e2bad7b348c53f","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc42edd949e2bad7b348c53f: start","job_status":"start","queue_duration_s":2.439862,"scheduling_latency_s":2.439879}1394gitlab # [ 240.855948] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.570Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc42edd949e2bad7b348c53f","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc42edd949e2bad7b348c53f: done: 0.014931 sec","job_status":"done","queue_duration_s":2.439862,"scheduling_latency_s":2.439879,"redis_calls":13,"redis_duration_s":0.001722,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000564,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001158,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010457,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014931,"completed_at":"2026-09-13T10:14:41.570Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1395gitlab # [ 240.894274] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.572Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"06a16d8490a62ee3c5f0205c","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-06a16d8490a62ee3c5f0205c: start","job_status":"start","queue_duration_s":2.455849,"scheduling_latency_s":2.455866}1396gitlab # [ 240.906481] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.586Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"06a16d8490a62ee3c5f0205c","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-06a16d8490a62ee3c5f0205c: done: 0.014784 sec","job_status":"done","queue_duration_s":2.455849,"scheduling_latency_s":2.455866,"redis_calls":13,"redis_duration_s":0.001528,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000586,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000942,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010476,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014784,"completed_at":"2026-09-13T10:14:41.586Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1397gitlab # [ 240.945585] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.587Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"288d691a1dfddbe600fcdd71","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-288d691a1dfddbe600fcdd71: start","job_status":"start","queue_duration_s":2.471661,"scheduling_latency_s":2.471675}1398gitlab # [ 240.957739] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.602Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"288d691a1dfddbe600fcdd71","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-288d691a1dfddbe600fcdd71: done: 0.014528 sec","job_status":"done","queue_duration_s":2.471661,"scheduling_latency_s":2.471675,"redis_calls":13,"redis_duration_s":0.001467,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000517,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00095,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009528,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014528,"completed_at":"2026-09-13T10:14:41.602Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1399gitlab # [ 240.995974] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.604Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d7769831f995d754a2d8c351","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d7769831f995d754a2d8c351: start","job_status":"start","queue_duration_s":2.487242,"scheduling_latency_s":2.487892}1400gitlab # [ 241.008092] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.619Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d7769831f995d754a2d8c351","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d7769831f995d754a2d8c351: done: 0.015959 sec","job_status":"done","queue_duration_s":2.487242,"scheduling_latency_s":2.487892,"redis_calls":13,"redis_duration_s":0.001513,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000548,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000965,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009609,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015959,"completed_at":"2026-09-13T10:14:41.619Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1401gitlab # [ 241.046125] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.620Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"96ed0b0b67493648a158ef7d","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-96ed0b0b67493648a158ef7d: start","job_status":"start","queue_duration_s":2.504314,"scheduling_latency_s":2.504329}1402gitlab # [ 241.058268] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.635Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"96ed0b0b67493648a158ef7d","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-96ed0b0b67493648a158ef7d: done: 0.015015 sec","job_status":"done","queue_duration_s":2.504314,"scheduling_latency_s":2.504329,"redis_calls":13,"redis_duration_s":0.0015869999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000584,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001003,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010688,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015015,"completed_at":"2026-09-13T10:14:41.635Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1403gitlab # [ 241.096403] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.636Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ac63975838a8bccd4df59586","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ac63975838a8bccd4df59586: start","job_status":"start","queue_duration_s":2.520408,"scheduling_latency_s":2.520423}1404gitlab # [ 241.108223] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.651Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ac63975838a8bccd4df59586","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ac63975838a8bccd4df59586: done: 0.015269 sec","job_status":"done","queue_duration_s":2.520408,"scheduling_latency_s":2.520423,"redis_calls":13,"redis_duration_s":0.001499,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000543,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000956,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01101,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015269,"completed_at":"2026-09-13T10:14:41.651Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1405gitlab # [ 241.147455] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.653Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3a2ad9a1b9d0d02837d35ad4","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3a2ad9a1b9d0d02837d35ad4: start","job_status":"start","queue_duration_s":2.536816,"scheduling_latency_s":2.536833}1406gitlab # [ 241.159711] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.668Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3a2ad9a1b9d0d02837d35ad4","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3a2ad9a1b9d0d02837d35ad4: done: 0.015763 sec","job_status":"done","queue_duration_s":2.536816,"scheduling_latency_s":2.536833,"redis_calls":13,"redis_duration_s":0.001483,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000545,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000938,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01121,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015763,"completed_at":"2026-09-13T10:14:41.668Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1407gitlab # [ 241.198484] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.669Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b67bae4373f4ab467cd0ff04","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b67bae4373f4ab467cd0ff04: start","job_status":"start","queue_duration_s":2.55364,"scheduling_latency_s":2.553656}1408gitlab # [ 241.210651] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.693Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b67bae4373f4ab467cd0ff04","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b67bae4373f4ab467cd0ff04: done: 0.023594 sec","job_status":"done","queue_duration_s":2.55364,"scheduling_latency_s":2.553656,"redis_calls":13,"redis_duration_s":0.0016420000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000599,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001043,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011703,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023594,"completed_at":"2026-09-13T10:14:41.693Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1409gitlab # [ 241.249321] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.694Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"89ab92ef1074d5d958f13dca","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-89ab92ef1074d5d958f13dca: start","job_status":"start","queue_duration_s":2.578347,"scheduling_latency_s":2.578363}1410gitlab # [ 241.261558] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.710Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"89ab92ef1074d5d958f13dca","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-89ab92ef1074d5d958f13dca: done: 0.015774 sec","job_status":"done","queue_duration_s":2.578347,"scheduling_latency_s":2.578363,"redis_calls":13,"redis_duration_s":0.002584,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002048,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011287,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015774,"completed_at":"2026-09-13T10:14:41.710Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1411gitlab # [ 241.299981] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.711Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0b7833b3b467c14bc704c040","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0b7833b3b467c14bc704c040: start","job_status":"start","queue_duration_s":2.59518,"scheduling_latency_s":2.595196}1412gitlab # [ 241.312209] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.725Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0b7833b3b467c14bc704c040","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0b7833b3b467c14bc704c040: done: 0.014369 sec","job_status":"done","queue_duration_s":2.59518,"scheduling_latency_s":2.595196,"redis_calls":13,"redis_duration_s":0.001461,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000542,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000919,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010091,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014369,"completed_at":"2026-09-13T10:14:41.725Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1413gitlab # [ 241.351375] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.726Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85157196e1340e52306ff689","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85157196e1340e52306ff689: start","job_status":"start","queue_duration_s":2.610657,"scheduling_latency_s":2.610674}1414gitlab # [ 241.363746] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.741Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85157196e1340e52306ff689","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85157196e1340e52306ff689: done: 0.014681 sec","job_status":"done","queue_duration_s":2.610657,"scheduling_latency_s":2.610674,"redis_calls":13,"redis_duration_s":0.001461,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000938,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009574,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014681,"completed_at":"2026-09-13T10:14:41.741Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1415gitlab # [ 241.402041] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.743Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fded9d1d14c2419750ab7d8c","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fded9d1d14c2419750ab7d8c: start","job_status":"start","queue_duration_s":2.626959,"scheduling_latency_s":2.626975}1416gitlab # [ 241.414320] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.757Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fded9d1d14c2419750ab7d8c","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fded9d1d14c2419750ab7d8c: done: 0.014484 sec","job_status":"done","queue_duration_s":2.626959,"scheduling_latency_s":2.626975,"redis_calls":13,"redis_duration_s":0.001465,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000547,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000918,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010148,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014484,"completed_at":"2026-09-13T10:14:41.757Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1417gitlab # [ 241.452719] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.758Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"73c31e306c1262ff1ced638f","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-73c31e306c1262ff1ced638f: start","job_status":"start","queue_duration_s":2.642491,"scheduling_latency_s":2.642507}1418gitlab # [ 241.465167] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.773Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"73c31e306c1262ff1ced638f","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-73c31e306c1262ff1ced638f: done: 0.01484 sec","job_status":"done","queue_duration_s":2.642491,"scheduling_latency_s":2.642507,"redis_calls":13,"redis_duration_s":0.001683,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000534,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001149,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010355,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01484,"completed_at":"2026-09-13T10:14:41.773Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1419gitlab # [ 241.504135] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.774Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5fbe4d0ccbed6091480be574","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5fbe4d0ccbed6091480be574: start","job_status":"start","queue_duration_s":2.658401,"scheduling_latency_s":2.658416}1420gitlab # [ 241.516324] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.789Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5fbe4d0ccbed6091480be574","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5fbe4d0ccbed6091480be574: done: 0.014502 sec","job_status":"done","queue_duration_s":2.658401,"scheduling_latency_s":2.658416,"redis_calls":13,"redis_duration_s":0.001464,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000932,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009602,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014502,"completed_at":"2026-09-13T10:14:41.789Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1421gitlab # [ 241.555353] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.790Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6447a29c7e13fb042e3227ed","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6447a29c7e13fb042e3227ed: start","job_status":"start","queue_duration_s":2.674556,"scheduling_latency_s":2.674572}1422gitlab # [ 241.567792] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.805Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6447a29c7e13fb042e3227ed","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6447a29c7e13fb042e3227ed: done: 0.014561 sec","job_status":"done","queue_duration_s":2.674556,"scheduling_latency_s":2.674572,"redis_calls":13,"redis_duration_s":0.002046,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001128,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000918,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010166,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014561,"completed_at":"2026-09-13T10:14:41.805Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1423gitlab # [ 241.606400] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.806Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"74b10f0b061d4459cdb7bcb9","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-74b10f0b061d4459cdb7bcb9: start","job_status":"start","queue_duration_s":2.690204,"scheduling_latency_s":2.69022}1424gitlab # [ 241.618508] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.821Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"74b10f0b061d4459cdb7bcb9","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-74b10f0b061d4459cdb7bcb9: done: 0.015081 sec","job_status":"done","queue_duration_s":2.690204,"scheduling_latency_s":2.69022,"redis_calls":13,"redis_duration_s":0.001519,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000563,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000956,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010624,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015081,"completed_at":"2026-09-13T10:14:41.821Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1425gitlab # [ 241.656889] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.822Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cabf9410ad60dd62256b363a","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cabf9410ad60dd62256b363a: start","job_status":"start","queue_duration_s":2.706279,"scheduling_latency_s":2.706295}1426gitlab # [ 241.669048] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.837Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cabf9410ad60dd62256b363a","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cabf9410ad60dd62256b363a: done: 0.015346 sec","job_status":"done","queue_duration_s":2.706279,"scheduling_latency_s":2.706295,"redis_calls":13,"redis_duration_s":0.001444,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000529,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000915,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009486,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015346,"completed_at":"2026-09-13T10:14:41.837Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1427gitlab # [ 241.707353] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.838Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cefc27820c5bbe78cf2ab726","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cefc27820c5bbe78cf2ab726: start","job_status":"start","queue_duration_s":2.72272,"scheduling_latency_s":2.722735}1428gitlab # [ 241.719706] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.853Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cefc27820c5bbe78cf2ab726","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cefc27820c5bbe78cf2ab726: done: 0.014493 sec","job_status":"done","queue_duration_s":2.72272,"scheduling_latency_s":2.722735,"redis_calls":13,"redis_duration_s":0.001497,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00097,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010206,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014493,"completed_at":"2026-09-13T10:14:41.853Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1429gitlab # [ 241.759137] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.854Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"101c1f5c05c385e85d2c8219","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-101c1f5c05c385e85d2c8219: start","job_status":"start","queue_duration_s":2.738273,"scheduling_latency_s":2.738289}1430gitlab # [ 241.771450] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.869Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"101c1f5c05c385e85d2c8219","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-101c1f5c05c385e85d2c8219: done: 0.015315 sec","job_status":"done","queue_duration_s":2.738273,"scheduling_latency_s":2.738289,"redis_calls":13,"redis_duration_s":0.001565,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000587,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000978,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010716,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015315,"completed_at":"2026-09-13T10:14:41.869Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1431gitlab # [ 241.809899] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.870Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d14292bc5ef91af4accb319e","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d14292bc5ef91af4accb319e: start","job_status":"start","queue_duration_s":2.754664,"scheduling_latency_s":2.75468}1432gitlab # [ 241.822165] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.885Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d14292bc5ef91af4accb319e","created_at":"2026-09-13T10:14:38.903Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-bce7961ac3ff4957","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"925207b9cb631ca8541d21ee5b74dc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.116Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d14292bc5ef91af4accb319e: done: 0.014355 sec","job_status":"done","queue_duration_s":2.754664,"scheduling_latency_s":2.75468,"redis_calls":13,"redis_duration_s":0.001465,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00094,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009468,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014355,"completed_at":"2026-09-13T10:14:41.885Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1433gitlab # [ 241.861519] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.886Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"527d59dc534ebef262b05a06","created_at":"2026-09-13T10:14:39.153Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-cf586b30c0c54f84","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"098c7b26431a0f611f4b224471d4c87c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.171Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-527d59dc534ebef262b05a06: start","job_status":"start","queue_duration_s":2.715414,"scheduling_latency_s":2.715431}1434gitlab # [ 241.873678] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.904Z","message":"No existing merge request to be cleaned up."}1435gitlab # [ 241.875347] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.919Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"527d59dc534ebef262b05a06","created_at":"2026-09-13T10:14:39.153Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-cf586b30c0c54f84","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"098c7b26431a0f611f4b224471d4c87c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.171Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-527d59dc534ebef262b05a06: done: 0.032505 sec","job_status":"done","queue_duration_s":2.715414,"scheduling_latency_s":2.715431,"redis_calls":13,"redis_duration_s":0.001777,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.0003,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00066,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000817,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01997,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025132,"duration_s":0.032505,"completed_at":"2026-09-13T10:14:41.919Z","load_balancing_strategy":"primary","db_duration_s":0.006358,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1436gitlab # [ 241.915489] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.920Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"03670d7b22cfbc6f830b366b","created_at":"2026-09-13T10:14:39.153Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-cf586b30c0c54f84","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"098c7b26431a0f611f4b224471d4c87c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.171Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-03670d7b22cfbc6f830b366b: start","job_status":"start","queue_duration_s":2.748985,"scheduling_latency_s":2.749002}1437gitlab # [ 241.927987] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.930Z","message":"No existing merge request to be cleaned up."}1438gitlab # [ 241.929613] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.938Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"03670d7b22cfbc6f830b366b","created_at":"2026-09-13T10:14:39.153Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-cf586b30c0c54f84","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"098c7b26431a0f611f4b224471d4c87c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.171Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-03670d7b22cfbc6f830b366b: done: 0.018516 sec","job_status":"done","queue_duration_s":2.748985,"scheduling_latency_s":2.749002,"redis_calls":10,"redis_duration_s":0.001349,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000581,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000768,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012788,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018516,"completed_at":"2026-09-13T10:14:41.938Z","load_balancing_strategy":"primary","db_duration_s":0.00251,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1439gitlab # [ 241.968660] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.939Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d213a2b2391b12585afee398","created_at":"2026-09-13T10:14:39.153Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-cf586b30c0c54f84","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"098c7b26431a0f611f4b224471d4c87c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.171Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d213a2b2391b12585afee398: start","job_status":"start","queue_duration_s":2.768527,"scheduling_latency_s":2.768544}1440gitlab # [ 241.980964] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.950Z","message":"No existing merge request to be cleaned up."}1441gitlab # [ 241.982581] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.958Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d213a2b2391b12585afee398","created_at":"2026-09-13T10:14:39.153Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-cf586b30c0c54f84","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"098c7b26431a0f611f4b224471d4c87c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.171Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d213a2b2391b12585afee398: done: 0.01891 sec","job_status":"done","queue_duration_s":2.768527,"scheduling_latency_s":2.768544,"redis_calls":10,"redis_duration_s":0.001336,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000582,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000754,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012763,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01891,"completed_at":"2026-09-13T10:14:41.958Z","load_balancing_strategy":"primary","db_duration_s":0.002401,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1442gitlab # [ 242.020421] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.959Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ee98ae6040545afdd8af43a2","created_at":"2026-09-13T10:14:39.153Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-cf586b30c0c54f84","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"098c7b26431a0f611f4b224471d4c87c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.171Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ee98ae6040545afdd8af43a2: start","job_status":"start","queue_duration_s":2.788501,"scheduling_latency_s":2.788518}1443gitlab # [ 242.032709] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.970Z","message":"No existing merge request to be cleaned up."}1444gitlab # [ 242.034371] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:14:41.979Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ee98ae6040545afdd8af43a2","created_at":"2026-09-13T10:14:39.153Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-cf586b30c0c54f84","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"098c7b26431a0f611f4b224471d4c87c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:14:39.171Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ee98ae6040545afdd8af43a2: done: 0.019092 sec","job_status":"done","queue_duration_s":2.788501,"scheduling_latency_s":2.788518,"redis_calls":10,"redis_duration_s":0.0013809999999999998,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000592,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000789,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013097,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019092,"completed_at":"2026-09-13T10:14:41.978Z","load_balancing_strategy":"primary","db_duration_s":0.002965,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1445gitlab # [ 262.289545] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:04.196Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"bd450df5d00f91ec67b60807","created_at":"2026-09-13T10:15:04.173Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"97af98de7f094d8fbcac1f44de1ed206","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.193Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-bd450df5d00f91ec67b60807: start","job_status":"start","queue_duration_s":0.003085,"scheduling_latency_s":0.003143}1446gitlab # [ 262.401156] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:04.307Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"bd450df5d00f91ec67b60807","created_at":"2026-09-13T10:15:04.173Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"97af98de7f094d8fbcac1f44de1ed206","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.193Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-bd450df5d00f91ec67b60807: done: 0.111095 sec","job_status":"done","queue_duration_s":0.003085,"scheduling_latency_s":0.003143,"redis_calls":11,"redis_duration_s":0.016417,"redis_read_bytes":9,"redis_write_bytes":1174,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001544,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.007577,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.005315,"redis_rate_limiting_write_bytes":51,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001981,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041192,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00335154,"extra.cells_stale_requests_cleanup_cron_worker.removed_count":0,"duration_s":0.111095,"completed_at":"2026-09-13T10:15:04.307Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001326,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1447gitlab # [ 262.492936] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:04.317Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"ff629df3ce4ca55e4b37172b","created_at":"2026-09-13T10:15:04.205Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"041f9a51c6aebd5f15c1776f9f720749","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.211Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-ff629df3ce4ca55e4b37172b: start","job_status":"start","queue_duration_s":0.106376,"scheduling_latency_s":0.106424}1448gitlab # [ 262.510172] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:04.381Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1449gitlab # [ 262.514282] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:04.409Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"ff629df3ce4ca55e4b37172b","created_at":"2026-09-13T10:15:04.205Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"041f9a51c6aebd5f15c1776f9f720749","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.211Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-ff629df3ce4ca55e4b37172b: done: 0.091724 sec","job_status":"done","queue_duration_s":0.106376,"scheduling_latency_s":0.106424,"redis_calls":7,"redis_duration_s":0.019344,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.014152,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.005192,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024396,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.091724,"completed_at":"2026-09-13T10:15:04.409Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1450gitlab # [ 262.554765] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:04.426Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"2fca3848b1f5cc6aa2ceba26","created_at":"2026-09-13T10:15:04.253Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8bf1b20bcb247c729862839a456bec30","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-13T10:15:04.264Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-2fca3848b1f5cc6aa2ceba26: start","job_status":"start","queue_duration_s":0.162732,"scheduling_latency_s":0.162783}1451gitlab # [ 262.605892] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:04.512Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"2fca3848b1f5cc6aa2ceba26","created_at":"2026-09-13T10:15:04.253Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8bf1b20bcb247c729862839a456bec30","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-13T10:15:04.264Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-2fca3848b1f5cc6aa2ceba26: done: 0.085406 sec","job_status":"done","queue_duration_s":0.162732,"scheduling_latency_s":0.162783,"redis_calls":7,"redis_duration_s":0.005374,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003042,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002332,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038733,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.085406,"completed_at":"2026-09-13T10:15:04.512Z","load_balancing_strategy":"primary","db_duration_s":0.016906,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1452gitlab # [ 262.664306] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:04.517Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"5ceaab423551f813030935d7","created_at":"2026-09-13T10:15:04.298Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1a1cfed0f3ebc7f1c0fed83cde1f36fa","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.304Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-5ceaab423551f813030935d7: start","job_status":"start","queue_duration_s":0.212921,"scheduling_latency_s":0.212967}1453gitlab # [ 262.679809] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:04.541Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1454gitlab # [ 262.699425] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:04.607Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"5ceaab423551f813030935d7","created_at":"2026-09-13T10:15:04.298Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1a1cfed0f3ebc7f1c0fed83cde1f36fa","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.304Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-5ceaab423551f813030935d7: done: 0.090432 sec","job_status":"done","queue_duration_s":0.212921,"scheduling_latency_s":0.212967,"redis_calls":7,"redis_duration_s":0.0022819999999999997,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001646,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000636,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020496,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.090432,"completed_at":"2026-09-13T10:15:04.607Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1455gitlab # [ 262.753287] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:04.643Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"9cca48a95692ab668122376f","created_at":"2026-09-13T10:15:04.343Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"36af1fd3052f555ce74ef00481beaf55","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-13T10:15:04.347Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-9cca48a95692ab668122376f: start","job_status":"start","queue_duration_s":0.295834,"scheduling_latency_s":0.295879}1456gitlab # [ 262.804098] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:04.709Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1457gitlab # [ 262.846099] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:04.754Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"9cca48a95692ab668122376f","created_at":"2026-09-13T10:15:04.343Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"36af1fd3052f555ce74ef00481beaf55","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-13T10:15:04.347Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-9cca48a95692ab668122376f: done: 0.110832 sec","job_status":"done","queue_duration_s":0.295834,"scheduling_latency_s":0.295879,"redis_calls":7,"redis_duration_s":0.0069299999999999995,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00537,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00156,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024042,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.110832,"completed_at":"2026-09-13T10:15:04.753Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1458gitlab # [ 262.912758] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:04.789Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"4ebbe98ae81b5b7c7af357ef","created_at":"2026-09-13T10:15:04.383Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"75a69050cdb11b4e517657d4defb1d3e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.395Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-4ebbe98ae81b5b7c7af357ef: start","job_status":"start","queue_duration_s":0.393865,"scheduling_latency_s":0.393907}1459gitlab # [ 263.032847] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:04.940Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"4ebbe98ae81b5b7c7af357ef","created_at":"2026-09-13T10:15:04.383Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"75a69050cdb11b4e517657d4defb1d3e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.395Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-4ebbe98ae81b5b7c7af357ef: done: 0.151324 sec","job_status":"done","queue_duration_s":0.393865,"scheduling_latency_s":0.393907,"redis_calls":7,"redis_duration_s":0.023315000000000002,"redis_read_bytes":5,"redis_write_bytes":776,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.020681,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002634,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02467,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.151324,"completed_at":"2026-09-13T10:15:04.940Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1460gitlab # [ 263.120547] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:04.944Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"3fe1cd221959042fd5904e17","created_at":"2026-09-13T10:15:04.428Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.437Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-3fe1cd221959042fd5904e17: start","job_status":"start","queue_duration_s":0.506635,"scheduling_latency_s":0.506684}1461gitlab # [ 263.184509] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:05.093Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"3fe1cd221959042fd5904e17","created_at":"2026-09-13T10:15:04.428Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.437Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-3fe1cd221959042fd5904e17: done: 0.14876 sec","job_status":"done","queue_duration_s":0.506635,"scheduling_latency_s":0.506684,"redis_calls":60,"redis_duration_s":0.018471,"redis_read_bytes":108,"redis_write_bytes":41059,"redis_queues_calls":2,"redis_queues_duration_s":0.000813,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.016716,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000942,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.130891,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.14876,"completed_at":"2026-09-13T10:15:05.093Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1462gitlab # [ 263.279620] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:05.094Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"b811b5d462bcdd5f61b13925","created_at":"2026-09-13T10:15:04.495Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8ff768b3cf9ca8b708f686af246e3983","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.557Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-b811b5d462bcdd5f61b13925: start","job_status":"start","queue_duration_s":0.537635,"scheduling_latency_s":0.537652}1463gitlab # [ 263.298669] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:05.131Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"b811b5d462bcdd5f61b13925","created_at":"2026-09-13T10:15:04.495Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8ff768b3cf9ca8b708f686af246e3983","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.557Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-b811b5d462bcdd5f61b13925: done: 0.037303 sec","job_status":"done","queue_duration_s":0.537635,"scheduling_latency_s":0.537652,"redis_calls":10,"redis_duration_s":0.00352,"redis_read_bytes":9,"redis_write_bytes":1071,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000602,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001712,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001206,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019519,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00054836,"duration_s":0.037303,"completed_at":"2026-09-13T10:15:05.131Z","load_balancing_strategy":"primary","db_duration_s":0.009035,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1464gitlab # [ 263.347507] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:05.134Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"4a9e2909497453875b093eab","created_at":"2026-09-13T10:15:04.577Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d01b9fcd3b06c6b9fdc844b41d1f4250","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.579Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-4a9e2909497453875b093eab: start","job_status":"start","queue_duration_s":0.554141,"scheduling_latency_s":0.55416}1465gitlab # [ 263.360764] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:05.173Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"4a9e2909497453875b093eab","created_at":"2026-09-13T10:15:04.577Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d01b9fcd3b06c6b9fdc844b41d1f4250","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.579Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-4a9e2909497453875b093eab: done: 0.039763 sec","job_status":"done","queue_duration_s":0.554141,"scheduling_latency_s":0.55416,"redis_calls":14,"redis_duration_s":0.0021910000000000002,"redis_read_bytes":16,"redis_write_bytes":4026,"redis_queues_calls":2,"redis_queues_duration_s":0.000234,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001427,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00053,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029555,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.039763,"completed_at":"2026-09-13T10:15:05.173Z","load_balancing_strategy":"primary","db_duration_s":0.004697,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1466gitlab # [ 263.401807] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:05.175Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"e689053c0bb61e67bc8639de","created_at":"2026-09-13T10:15:04.583Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"567ba4d434f2e644324fe62dca93e47a","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.586Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-e689053c0bb61e67bc8639de: start","job_status":"start","queue_duration_s":0.588358,"scheduling_latency_s":0.588375}1467gitlab # [ 263.567441] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:05.476Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"e689053c0bb61e67bc8639de","created_at":"2026-09-13T10:15:04.583Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"567ba4d434f2e644324fe62dca93e47a","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.586Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-e689053c0bb61e67bc8639de: done: 0.300976 sec","job_status":"done","queue_duration_s":0.588358,"scheduling_latency_s":0.588375,"redis_calls":562,"redis_duration_s":0.058542,"redis_read_bytes":560,"redis_write_bytes":48760,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000768,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.057774,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.276856,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.300976,"completed_at":"2026-09-13T10:15:05.475Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1468gitlab # [ 263.662663] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:05.477Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"3f76390be95010d00140e4b3","created_at":"2026-09-13T10:15:04.589Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"81af22f15197226863014b17aae94b60","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-13T10:15:04.593Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-3f76390be95010d00140e4b3: start","job_status":"start","queue_duration_s":0.883907,"scheduling_latency_s":0.883931}1469gitlab # [ 263.680142] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:05.485Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1470gitlab # [ 263.684355] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:05.490Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"3f76390be95010d00140e4b3","created_at":"2026-09-13T10:15:04.589Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"81af22f15197226863014b17aae94b60","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-13T10:15:04.593Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-3f76390be95010d00140e4b3: done: 0.01326 sec","job_status":"done","queue_duration_s":0.883907,"scheduling_latency_s":0.883931,"redis_calls":7,"redis_duration_s":0.001101,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000782,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000319,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008304,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01326,"completed_at":"2026-09-13T10:15:05.490Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1471gitlab # [ 263.724341] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:05.492Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"ef6b18d776372ce55655dcb2","created_at":"2026-09-13T10:15:04.601Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2f65ec3bda58a939706b971f002f76cf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.604Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-ef6b18d776372ce55655dcb2: start","job_status":"start","queue_duration_s":0.887737,"scheduling_latency_s":0.887755}1472gitlab # [ 263.738426] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:05.607Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"ef6b18d776372ce55655dcb2","created_at":"2026-09-13T10:15:04.601Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2f65ec3bda58a939706b971f002f76cf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.604Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-ef6b18d776372ce55655dcb2: done: 0.115533 sec","job_status":"done","queue_duration_s":0.887737,"scheduling_latency_s":0.887755,"redis_calls":9,"redis_duration_s":0.002584,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00129,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001294,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.02,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.044472,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00054358,"exclusive_lock_hold_duration_s":0.09366606,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.115533,"completed_at":"2026-09-13T10:15:05.607Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.07341,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1473gitlab # [ 263.780271] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:05.610Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"27fadab79b9e0a17ddb0c4d5","created_at":"2026-09-13T10:15:04.614Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a97823e82ea3f7e9185e01470458be06","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.618Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-27fadab79b9e0a17ddb0c4d5: start","job_status":"start","queue_duration_s":0.992424,"scheduling_latency_s":0.99246}1474gitlab # [ 263.795580] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:05.625Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1475gitlab # [ 263.798857] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:05.635Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"27fadab79b9e0a17ddb0c4d5","created_at":"2026-09-13T10:15:04.614Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a97823e82ea3f7e9185e01470458be06","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.618Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-27fadab79b9e0a17ddb0c4d5: done: 0.024654 sec","job_status":"done","queue_duration_s":0.992424,"scheduling_latency_s":0.99246,"redis_calls":7,"redis_duration_s":0.002867,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002121,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000746,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015031,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024654,"completed_at":"2026-09-13T10:15:05.635Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1476gitlab # [ 263.838791] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:05.637Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Gitlab::Import::StuckProjectImportJobsWorker","jid":"3fe0ca1a1443bd9b20ce61ec","created_at":"2026-09-13T10:15:04.626Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"93d18bded111877d8f264f0028053de7","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:65d1a1bda9b2a0115065de4f20868193e50a174305a7166fa20fbc82759e43e4","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.635Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Gitlab::Import::StuckProjectImportJobsWorker JID-3fe0ca1a1443bd9b20ce61ec: start","job_status":"start","queue_duration_s":1.001598,"scheduling_latency_s":1.001624}1477gitlab # [ 263.852088] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:05.694Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Gitlab::Import::StuckProjectImportJobsWorker","jid":"3fe0ca1a1443bd9b20ce61ec","created_at":"2026-09-13T10:15:04.626Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"93d18bded111877d8f264f0028053de7","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:65d1a1bda9b2a0115065de4f20868193e50a174305a7166fa20fbc82759e43e4","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.635Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Gitlab::Import::StuckProjectImportJobsWorker JID-3fe0ca1a1443bd9b20ce61ec: done: 0.056847 sec","job_status":"done","queue_duration_s":1.001598,"scheduling_latency_s":1.001624,"redis_calls":10,"redis_duration_s":0.0035150000000000003,"redis_read_bytes":9,"redis_write_bytes":1173,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000756,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002076,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000683,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":236,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03758,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00067104,"duration_s":0.056847,"completed_at":"2026-09-13T10:15:05.694Z","load_balancing_strategy":"primary","db_duration_s":0.006661,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1478gitlab # [ 263.892325] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:05.696Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckMergeJobsWorker","jid":"22d6826ef32bbf51b6a1b928","created_at":"2026-09-13T10:15:04.642Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2e7ae965092f6b58687c072c52a3a374","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:ce304458c2d63d5d84855b65f51692704db52bf763a718b10c08a2bd5f588f06","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.651Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckMergeJobsWorker JID-22d6826ef32bbf51b6a1b928: start","job_status":"start","queue_duration_s":1.045279,"scheduling_latency_s":1.045317}1479gitlab # [ 263.905237] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:05.762Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckMergeJobsWorker","jid":"22d6826ef32bbf51b6a1b928","created_at":"2026-09-13T10:15:04.642Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2e7ae965092f6b58687c072c52a3a374","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:ce304458c2d63d5d84855b65f51692704db52bf763a718b10c08a2bd5f588f06","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.651Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckMergeJobsWorker JID-22d6826ef32bbf51b6a1b928: done: 0.065433 sec","job_status":"done","queue_duration_s":1.045279,"scheduling_latency_s":1.045317,"redis_calls":11,"redis_duration_s":0.004024,"redis_read_bytes":10,"redis_write_bytes":1022,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000572,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":283,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002319,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001133,"redis_shared_state_read_bytes":4,"redis_shared_state_write_bytes":225,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03589,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00097454,"duration_s":0.065433,"completed_at":"2026-09-13T10:15:05.762Z","load_balancing_strategy":"primary","db_duration_s":0.014051,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1480gitlab # [ 263.945022] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:05.766Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainVerificationCronWorker","jid":"965d3479b92dd5ea3391a4e9","created_at":"2026-09-13T10:15:04.665Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8eafabac1e3f6e5c955d881b0d64260b","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:4b3095972298b58b9c376f502255d63e0934f66e5921dfe9547414a5f347be92","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.672Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainVerificationCronWorker JID-965d3479b92dd5ea3391a4e9: start","job_status":"start","queue_duration_s":1.093103,"scheduling_latency_s":1.093145}1481gitlab # [ 263.957935] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:05.833Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainVerificationCronWorker","jid":"965d3479b92dd5ea3391a4e9","created_at":"2026-09-13T10:15:04.665Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8eafabac1e3f6e5c955d881b0d64260b","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:4b3095972298b58b9c376f502255d63e0934f66e5921dfe9547414a5f347be92","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.672Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainVerificationCronWorker JID-965d3479b92dd5ea3391a4e9: done: 0.067633 sec","job_status":"done","queue_duration_s":1.093103,"scheduling_latency_s":1.093145,"redis_calls":10,"redis_duration_s":0.004309,"redis_read_bytes":9,"redis_write_bytes":1097,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000678,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":309,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002673,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000958,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":218,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043515,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00102214,"duration_s":0.067633,"completed_at":"2026-09-13T10:15:05.833Z","load_balancing_strategy":"primary","db_duration_s":0.008186,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1482gitlab # [ 263.999412] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:05.837Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"f4ef9ac91440052b498471ec","created_at":"2026-09-13T10:15:04.675Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"50cf7fbc577723462b87c9b8fc69c221","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.679Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-f4ef9ac91440052b498471ec: start","job_status":"start","queue_duration_s":1.158058,"scheduling_latency_s":1.1581}1483gitlab # [ 264.014434] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:05.866Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"f4ef9ac91440052b498471ec","created_at":"2026-09-13T10:15:04.675Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"50cf7fbc577723462b87c9b8fc69c221","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.679Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-f4ef9ac91440052b498471ec: done: 0.028845 sec","job_status":"done","queue_duration_s":1.158058,"scheduling_latency_s":1.1581,"redis_calls":7,"redis_duration_s":0.002425,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001764,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000661,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018218,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028845,"completed_at":"2026-09-13T10:15:05.865Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1484gitlab # [ 264.055195] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:05.868Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMigrateExternalDiffsWorker","jid":"ae6c590cc1a325f492bae823","created_at":"2026-09-13T10:15:04.684Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"13e34d613c642cb02616dcea6078696b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1f56d9ee4ddd7fd390ae51dd41358b9c806acf2491cad4e28b1956ebf69a9b52","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.691Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMigrateExternalDiffsWorker JID-ae6c590cc1a325f492bae823: start","job_status":"start","queue_duration_s":1.177164,"scheduling_latency_s":1.177194}1485gitlab # [ 264.068460] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:05.909Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMigrateExternalDiffsWorker","jid":"ae6c590cc1a325f492bae823","created_at":"2026-09-13T10:15:04.684Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"13e34d613c642cb02616dcea6078696b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1f56d9ee4ddd7fd390ae51dd41358b9c806acf2491cad4e28b1956ebf69a9b52","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.691Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMigrateExternalDiffsWorker JID-ae6c590cc1a325f492bae823: done: 0.040487 sec","job_status":"done","queue_duration_s":1.177164,"scheduling_latency_s":1.177194,"redis_calls":12,"redis_duration_s":0.004006,"redis_read_bytes":11,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000725,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":311,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001861,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00142,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":533,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025165,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00121332,"exclusive_lock_hold_duration_s":0.0009563,"duration_s":0.040487,"completed_at":"2026-09-13T10:15:05.908Z","load_balancing_strategy":"primary","db_duration_s":0.000898,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1486gitlab # [ 264.109443] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:05.912Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"5a1c2e58bf7c86f0c11335b7","created_at":"2026-09-13T10:15:04.723Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2c31e516ed8c3b015d1ce37b00696b86","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.731Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-5a1c2e58bf7c86f0c11335b7: start","job_status":"start","queue_duration_s":1.180802,"scheduling_latency_s":1.180842}1487gitlab # [ 264.123476] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:05.974Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"5a1c2e58bf7c86f0c11335b7","created_at":"2026-09-13T10:15:04.723Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2c31e516ed8c3b015d1ce37b00696b86","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.731Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-5a1c2e58bf7c86f0c11335b7: done: 0.062872 sec","job_status":"done","queue_duration_s":1.180802,"scheduling_latency_s":1.180842,"redis_calls":7,"redis_duration_s":0.003793,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002874,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000919,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033241,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.062872,"completed_at":"2026-09-13T10:15:05.974Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.018194,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1488gitlab # [ 264.163758] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:05.978Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"a066668039f31a3749b9710e","created_at":"2026-09-13T10:15:04.749Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"15bf6443322561d7f21957124607fdb8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.758Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-a066668039f31a3749b9710e: start","job_status":"start","queue_duration_s":1.220697,"scheduling_latency_s":1.220747}1489gitlab # [ 264.177280] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.019Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"a066668039f31a3749b9710e","created_at":"2026-09-13T10:15:04.749Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"15bf6443322561d7f21957124607fdb8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.758Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-a066668039f31a3749b9710e: done: 0.0406 sec","job_status":"done","queue_duration_s":1.220697,"scheduling_latency_s":1.220747,"redis_calls":9,"redis_duration_s":0.005642,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001867,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003775,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024578,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00102234,"exclusive_lock_hold_duration_s":0.00710436,"duration_s":0.0406,"completed_at":"2026-09-13T10:15:06.019Z","load_balancing_strategy":"primary","db_duration_s":0.002446,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1490gitlab # [ 264.218123] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.022Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"a7bcf71fbbcd0a1edec076a1","created_at":"2026-09-13T10:15:04.768Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"abb375825d937eddc54ea1a767ff358c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.774Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-a7bcf71fbbcd0a1edec076a1: start","job_status":"start","queue_duration_s":1.247638,"scheduling_latency_s":1.247674}1491gitlab # [ 264.428554] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.335Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"a7bcf71fbbcd0a1edec076a1","created_at":"2026-09-13T10:15:04.768Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"abb375825d937eddc54ea1a767ff358c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.774Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-a7bcf71fbbcd0a1edec076a1: done: 0.31383 sec","job_status":"done","queue_duration_s":1.247638,"scheduling_latency_s":1.247674,"redis_calls":9,"redis_duration_s":0.00353,"redis_read_bytes":7,"redis_write_bytes":987,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001595,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001935,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":129,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":129,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.051,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.051,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.247158,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00086094,"exclusive_lock_hold_duration_s":0.28917366,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.31383,"completed_at":"2026-09-13T10:15:06.335Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.077265,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1492gitlab # [ 264.517273] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.337Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"d8ca08dfcf97ee250260aae2","created_at":"2026-09-13T10:15:04.788Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"63f4607ffe5ad43523e09143314d5359","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.795Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-d8ca08dfcf97ee250260aae2: start","job_status":"start","queue_duration_s":1.542672,"scheduling_latency_s":1.542697}1493gitlab # [ 264.533791] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.355Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"d8ca08dfcf97ee250260aae2","created_at":"2026-09-13T10:15:04.788Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"63f4607ffe5ad43523e09143314d5359","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.795Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-d8ca08dfcf97ee250260aae2: done: 0.017344 sec","job_status":"done","queue_duration_s":1.542672,"scheduling_latency_s":1.542697,"redis_calls":7,"redis_duration_s":0.001857,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001349,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000508,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009591,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017344,"completed_at":"2026-09-13T10:15:06.355Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001773,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1494gitlab # [ 264.574302] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.356Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"9e567321f01bd6b5aafaf427","created_at":"2026-09-13T10:15:04.823Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"36e5d5946f3622453f01d75e6eb0b4b8","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.826Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-9e567321f01bd6b5aafaf427: start","job_status":"start","queue_duration_s":1.530199,"scheduling_latency_s":1.530223}1495gitlab # [ 264.588029] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.390Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"9e567321f01bd6b5aafaf427","created_at":"2026-09-13T10:15:04.823Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"36e5d5946f3622453f01d75e6eb0b4b8","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.826Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-9e567321f01bd6b5aafaf427: done: 0.033575 sec","job_status":"done","queue_duration_s":1.530199,"scheduling_latency_s":1.530223,"redis_calls":7,"redis_duration_s":0.001943,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001404,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000539,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00982,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.033575,"completed_at":"2026-09-13T10:15:06.390Z","load_balancing_strategy":"primary","db_duration_s":0.017699,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1496gitlab # [ 264.629071] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.392Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"d1b733d0ae949c2c6599fa1b","created_at":"2026-09-13T10:15:04.830Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ff7b0177186e3368c2a93657ab406ac8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.833Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-d1b733d0ae949c2c6599fa1b: start","job_status":"start","queue_duration_s":1.558522,"scheduling_latency_s":1.558545}1497gitlab # [ 264.643507] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.442Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"d1b733d0ae949c2c6599fa1b","created_at":"2026-09-13T10:15:04.830Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ff7b0177186e3368c2a93657ab406ac8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.833Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-d1b733d0ae949c2c6599fa1b: done: 0.050673 sec","job_status":"done","queue_duration_s":1.558522,"scheduling_latency_s":1.558545,"redis_calls":9,"redis_duration_s":0.00276,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001321,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001439,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019087,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0008261,"exclusive_lock_hold_duration_s":0.02485774,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.050673,"completed_at":"2026-09-13T10:15:06.442Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.025922,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1498gitlab # [ 264.686147] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.445Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"53763b1f9b27a94d1d682a45","created_at":"2026-09-13T10:15:04.836Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b414367b8b5828b5f0321b492948e3ab","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.839Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-53763b1f9b27a94d1d682a45: start","job_status":"start","queue_duration_s":1.605472,"scheduling_latency_s":1.605499}1499gitlab # [ 264.700314] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.466Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"53763b1f9b27a94d1d682a45","created_at":"2026-09-13T10:15:04.836Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b414367b8b5828b5f0321b492948e3ab","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.839Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-53763b1f9b27a94d1d682a45: done: 0.021178 sec","job_status":"done","queue_duration_s":1.605472,"scheduling_latency_s":1.605499,"redis_calls":9,"redis_duration_s":0.002558,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001371,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001187,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01218,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0006224,"exclusive_lock_hold_duration_s":0.00397176,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.021178,"completed_at":"2026-09-13T10:15:06.466Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002084,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1500gitlab # [ 264.741022] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.467Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"88e67198a2408318f4bfd4db","created_at":"2026-09-13T10:15:04.847Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0520ef22f0fb0c02ecb05eff31bcc05a","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.852Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-88e67198a2408318f4bfd4db: start","job_status":"start","queue_duration_s":1.615308,"scheduling_latency_s":1.615332}1501gitlab # [ 264.755681] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.494Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"88e67198a2408318f4bfd4db","created_at":"2026-09-13T10:15:04.847Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0520ef22f0fb0c02ecb05eff31bcc05a","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.852Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-88e67198a2408318f4bfd4db: done: 0.026452 sec","job_status":"done","queue_duration_s":1.615308,"scheduling_latency_s":1.615332,"redis_calls":10,"redis_duration_s":0.00348,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001758,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001722,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015449,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00665758,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.026452,"completed_at":"2026-09-13T10:15:06.494Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003923,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1502gitlab # [ 264.797853] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.496Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"1118d666d279c6b9d1257b36","created_at":"2026-09-13T10:15:04.866Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1594e2d193d7ac5f7146a1f901f57023","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.869Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-1118d666d279c6b9d1257b36: start","job_status":"start","queue_duration_s":1.626141,"scheduling_latency_s":1.626161}1503gitlab # [ 264.812950] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.504Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1504gitlab # [ 264.816453] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.510Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"1118d666d279c6b9d1257b36","created_at":"2026-09-13T10:15:04.866Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1594e2d193d7ac5f7146a1f901f57023","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.869Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-1118d666d279c6b9d1257b36: done: 0.014243 sec","job_status":"done","queue_duration_s":1.626141,"scheduling_latency_s":1.626161,"redis_calls":7,"redis_duration_s":0.0016779999999999998,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00121,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000468,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008178,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014243,"completed_at":"2026-09-13T10:15:06.510Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1505gitlab # [ 264.856295] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.511Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Issues::RescheduleStuckIssueRebalancesWorker","jid":"f7d527af2b5d798371a1e0fd","created_at":"2026-09-13T10:15:04.900Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3b1f3d559f727a82eb136f929128d941","meta.root_caller_id":"Cronjob","meta.feature_category":"team_planning","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fc2fa70b35c572764e1a8fbcbc4fdd3df48ec5a5855870d65915560d23915134","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.919Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Issues::RescheduleStuckIssueRebalancesWorker JID-f7d527af2b5d798371a1e0fd: start","job_status":"start","queue_duration_s":1.592475,"scheduling_latency_s":1.592493}1506gitlab # [ 264.870357] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.531Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Issues::RescheduleStuckIssueRebalancesWorker","jid":"f7d527af2b5d798371a1e0fd","created_at":"2026-09-13T10:15:04.900Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3b1f3d559f727a82eb136f929128d941","meta.root_caller_id":"Cronjob","meta.feature_category":"team_planning","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fc2fa70b35c572764e1a8fbcbc4fdd3df48ec5a5855870d65915560d23915134","size_limiter":"validated","enqueued_at":"2026-09-13T10:15:04.919Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Issues::RescheduleStuckIssueRebalancesWorker JID-f7d527af2b5d798371a1e0fd: done: 0.019652 sec","job_status":"done","queue_duration_s":1.592475,"scheduling_latency_s":1.592493,"redis_calls":11,"redis_duration_s":0.002676,"redis_read_bytes":9,"redis_write_bytes":1239,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000551,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001404,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":610,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000721,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":298,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010809,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00032878,"duration_s":0.019652,"completed_at":"2026-09-13T10:15:06.531Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000671,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1507gitlab # [ 264.911568] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.533Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1e5880a6add4f9484733846c","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1e5880a6add4f9484733846c: start","job_status":"start","queue_duration_s":1.448362,"scheduling_latency_s":1.448381}1508gitlab # [ 264.923914] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.549Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1e5880a6add4f9484733846c","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1e5880a6add4f9484733846c: done: 0.016355 sec","job_status":"done","queue_duration_s":1.448362,"scheduling_latency_s":1.448381,"redis_calls":13,"redis_duration_s":0.002812,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00094,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001872,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009626,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016355,"completed_at":"2026-09-13T10:15:06.549Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1509gitlab # [ 264.962712] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.551Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a9a6b6f32c80cc94ee1339b5","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a9a6b6f32c80cc94ee1339b5: start","job_status":"start","queue_duration_s":1.466259,"scheduling_latency_s":1.466276}1510gitlab # [ 264.974906] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.567Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a9a6b6f32c80cc94ee1339b5","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a9a6b6f32c80cc94ee1339b5: done: 0.016047 sec","job_status":"done","queue_duration_s":1.466259,"scheduling_latency_s":1.466276,"redis_calls":13,"redis_duration_s":0.002968,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000987,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001981,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009538,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016047,"completed_at":"2026-09-13T10:15:06.567Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1511gitlab # [ 265.015124] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.568Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b82909c0e901e9f1d06a4d9b","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b82909c0e901e9f1d06a4d9b: start","job_status":"start","queue_duration_s":1.483835,"scheduling_latency_s":1.483852}1512gitlab # [ 265.027571] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.583Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b82909c0e901e9f1d06a4d9b","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b82909c0e901e9f1d06a4d9b: done: 0.015216 sec","job_status":"done","queue_duration_s":1.483835,"scheduling_latency_s":1.483852,"redis_calls":13,"redis_duration_s":0.002418,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001257,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001161,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009767,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015216,"completed_at":"2026-09-13T10:15:06.583Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1513gitlab # [ 265.065799] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.585Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad2d28a3e83c176753e829c3","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad2d28a3e83c176753e829c3: start","job_status":"start","queue_duration_s":1.500352,"scheduling_latency_s":1.500372}1514gitlab # [ 265.078162] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.599Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad2d28a3e83c176753e829c3","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad2d28a3e83c176753e829c3: done: 0.014542 sec","job_status":"done","queue_duration_s":1.500352,"scheduling_latency_s":1.500372,"redis_calls":13,"redis_duration_s":0.001511,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00098,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009864,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014542,"completed_at":"2026-09-13T10:15:06.599Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1515gitlab # [ 265.116836] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.601Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"79bb018627661fc8eda23833","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-79bb018627661fc8eda23833: start","job_status":"start","queue_duration_s":1.516126,"scheduling_latency_s":1.516146}1516gitlab # [ 265.128998] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.615Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"79bb018627661fc8eda23833","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-79bb018627661fc8eda23833: done: 0.014511 sec","job_status":"done","queue_duration_s":1.516126,"scheduling_latency_s":1.516146,"redis_calls":13,"redis_duration_s":0.0015270000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000562,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000965,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009825,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014511,"completed_at":"2026-09-13T10:15:06.615Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1517gitlab # [ 265.167325] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.616Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d92a3b4d5c0268af3736ff59","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d92a3b4d5c0268af3736ff59: start","job_status":"start","queue_duration_s":1.531969,"scheduling_latency_s":1.531988}1518gitlab # [ 265.179651] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.631Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d92a3b4d5c0268af3736ff59","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d92a3b4d5c0268af3736ff59: done: 0.014691 sec","job_status":"done","queue_duration_s":1.531969,"scheduling_latency_s":1.531988,"redis_calls":13,"redis_duration_s":0.001627,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000557,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00107,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009844,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014691,"completed_at":"2026-09-13T10:15:06.631Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1519gitlab # [ 265.219488] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.632Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9dbef618858e98def1d8c482","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9dbef618858e98def1d8c482: start","job_status":"start","queue_duration_s":1.547898,"scheduling_latency_s":1.547917}1520gitlab # [ 265.232036] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.647Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9dbef618858e98def1d8c482","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9dbef618858e98def1d8c482: done: 0.01436 sec","job_status":"done","queue_duration_s":1.547898,"scheduling_latency_s":1.547917,"redis_calls":13,"redis_duration_s":0.001507,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00097,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009711,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01436,"completed_at":"2026-09-13T10:15:06.647Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1521gitlab # [ 265.270605] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.648Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c52a44c13db1f232cb61955a","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c52a44c13db1f232cb61955a: start","job_status":"start","queue_duration_s":1.563528,"scheduling_latency_s":1.563547}1522gitlab # [ 265.283016] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.662Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c52a44c13db1f232cb61955a","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c52a44c13db1f232cb61955a: done: 0.014559 sec","job_status":"done","queue_duration_s":1.563528,"scheduling_latency_s":1.563547,"redis_calls":13,"redis_duration_s":0.001571,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000562,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001009,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009857,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014559,"completed_at":"2026-09-13T10:15:06.662Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1523gitlab # [ 265.321279] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.664Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cd5e8a8b18d9d33b779695ad","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cd5e8a8b18d9d33b779695ad: start","job_status":"start","queue_duration_s":1.579316,"scheduling_latency_s":1.579366}1524gitlab # [ 265.333443] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.678Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cd5e8a8b18d9d33b779695ad","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cd5e8a8b18d9d33b779695ad: done: 0.014663 sec","job_status":"done","queue_duration_s":1.579316,"scheduling_latency_s":1.579366,"redis_calls":13,"redis_duration_s":0.001722,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001183,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00976,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014663,"completed_at":"2026-09-13T10:15:06.678Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1525gitlab # [ 265.372772] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.679Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3decaf7361217c13bc977558","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3decaf7361217c13bc977558: start","job_status":"start","queue_duration_s":1.594993,"scheduling_latency_s":1.595009}1526gitlab # [ 265.385042] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.693Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3decaf7361217c13bc977558","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3decaf7361217c13bc977558: done: 0.013911 sec","job_status":"done","queue_duration_s":1.594993,"scheduling_latency_s":1.595009,"redis_calls":13,"redis_duration_s":0.0014520000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000928,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009641,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013911,"completed_at":"2026-09-13T10:15:06.693Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1527gitlab # [ 265.424830] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.694Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"27bc5dff85f64e7e25de37a4","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-27bc5dff85f64e7e25de37a4: start","job_status":"start","queue_duration_s":1.610014,"scheduling_latency_s":1.610029}1528gitlab # [ 265.437414] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.708Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"27bc5dff85f64e7e25de37a4","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-27bc5dff85f64e7e25de37a4: done: 0.013918 sec","job_status":"done","queue_duration_s":1.610014,"scheduling_latency_s":1.610029,"redis_calls":13,"redis_duration_s":0.001466,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000929,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009604,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013918,"completed_at":"2026-09-13T10:15:06.708Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1529gitlab # [ 265.476099] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.709Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0f8f911ae2d94c69850d798e","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0f8f911ae2d94c69850d798e: start","job_status":"start","queue_duration_s":1.625048,"scheduling_latency_s":1.625065}1530gitlab # [ 265.488368] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.729Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0f8f911ae2d94c69850d798e","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0f8f911ae2d94c69850d798e: done: 0.019976 sec","job_status":"done","queue_duration_s":1.625048,"scheduling_latency_s":1.625065,"redis_calls":13,"redis_duration_s":0.0057090000000000005,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004714,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000995,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009972,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019976,"completed_at":"2026-09-13T10:15:06.729Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1531gitlab # [ 265.526931] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.731Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a611a8993530bfc4f8655f1a","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a611a8993530bfc4f8655f1a: start","job_status":"start","queue_duration_s":1.646118,"scheduling_latency_s":1.646135}1532gitlab # [ 265.539100] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.745Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a611a8993530bfc4f8655f1a","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a611a8993530bfc4f8655f1a: done: 0.014072 sec","job_status":"done","queue_duration_s":1.646118,"scheduling_latency_s":1.646135,"redis_calls":13,"redis_duration_s":0.00149,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000549,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000941,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009702,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014072,"completed_at":"2026-09-13T10:15:06.744Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1533gitlab # [ 265.577411] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.746Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"580563dbec0ae74708b2458e","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-580563dbec0ae74708b2458e: start","job_status":"start","queue_duration_s":1.661272,"scheduling_latency_s":1.661288}1534gitlab # [ 265.589657] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.759Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"580563dbec0ae74708b2458e","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-580563dbec0ae74708b2458e: done: 0.013763 sec","job_status":"done","queue_duration_s":1.661272,"scheduling_latency_s":1.661288,"redis_calls":13,"redis_duration_s":0.001456,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000933,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009521,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013763,"completed_at":"2026-09-13T10:15:06.759Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1535gitlab # [ 265.629536] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.761Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"66c407319cedf545cd61e878","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-66c407319cedf545cd61e878: start","job_status":"start","queue_duration_s":1.676211,"scheduling_latency_s":1.676228}1536gitlab # [ 265.642190] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.774Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"66c407319cedf545cd61e878","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-66c407319cedf545cd61e878: done: 0.0139 sec","job_status":"done","queue_duration_s":1.676211,"scheduling_latency_s":1.676228,"redis_calls":13,"redis_duration_s":0.001509,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000535,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000974,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009624,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0139,"completed_at":"2026-09-13T10:15:06.774Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1537gitlab # [ 265.681038] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.776Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"475906c828e97c3f9dbb4ad3","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-475906c828e97c3f9dbb4ad3: start","job_status":"start","queue_duration_s":1.691452,"scheduling_latency_s":1.69147}1538gitlab # [ 265.693156] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.790Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"475906c828e97c3f9dbb4ad3","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-475906c828e97c3f9dbb4ad3: done: 0.013869 sec","job_status":"done","queue_duration_s":1.691452,"scheduling_latency_s":1.69147,"redis_calls":13,"redis_duration_s":0.0014689999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00052,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000949,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009543,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013869,"completed_at":"2026-09-13T10:15:06.790Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1539gitlab # [ 265.732149] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.791Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"73b53054ba3fa0d2e9a9ff3e","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-73b53054ba3fa0d2e9a9ff3e: start","job_status":"start","queue_duration_s":1.706359,"scheduling_latency_s":1.706376}1540gitlab # [ 265.744394] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.805Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"73b53054ba3fa0d2e9a9ff3e","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-73b53054ba3fa0d2e9a9ff3e: done: 0.013811 sec","job_status":"done","queue_duration_s":1.706359,"scheduling_latency_s":1.706376,"redis_calls":13,"redis_duration_s":0.001436,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000517,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000919,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009549,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013811,"completed_at":"2026-09-13T10:15:06.804Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1541gitlab # [ 265.783286] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.806Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8e621a6144edeaebd8f2dec6","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8e621a6144edeaebd8f2dec6: start","job_status":"start","queue_duration_s":1.721256,"scheduling_latency_s":1.721273}1542gitlab # [ 265.795754] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.819Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8e621a6144edeaebd8f2dec6","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8e621a6144edeaebd8f2dec6: done: 0.013712 sec","job_status":"done","queue_duration_s":1.721256,"scheduling_latency_s":1.721273,"redis_calls":13,"redis_duration_s":0.001437,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000906,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009517,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013712,"completed_at":"2026-09-13T10:15:06.819Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1543gitlab # [ 265.836179] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.820Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"37353cb870d8b532328b4511","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-37353cb870d8b532328b4511: start","job_status":"start","queue_duration_s":1.736073,"scheduling_latency_s":1.73609}1544gitlab # [ 265.848593] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.834Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"37353cb870d8b532328b4511","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-37353cb870d8b532328b4511: done: 0.013873 sec","job_status":"done","queue_duration_s":1.736073,"scheduling_latency_s":1.73609,"redis_calls":13,"redis_duration_s":0.001437,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000518,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000919,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009665,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013873,"completed_at":"2026-09-13T10:15:06.834Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1545gitlab # [ 265.886682] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.835Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"93675a296f8ddfbead1bd123","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-93675a296f8ddfbead1bd123: start","job_status":"start","queue_duration_s":1.751036,"scheduling_latency_s":1.751053}1546gitlab # [ 265.898999] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.849Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"93675a296f8ddfbead1bd123","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-93675a296f8ddfbead1bd123: done: 0.013869 sec","job_status":"done","queue_duration_s":1.751036,"scheduling_latency_s":1.751053,"redis_calls":13,"redis_duration_s":0.001428,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000907,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009595,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013869,"completed_at":"2026-09-13T10:15:06.849Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1547gitlab # [ 265.937154] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.850Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d47c91d96c30eb4b07599b9c","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d47c91d96c30eb4b07599b9c: start","job_status":"start","queue_duration_s":1.766034,"scheduling_latency_s":1.766051}1548gitlab # [ 265.949241] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.864Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d47c91d96c30eb4b07599b9c","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d47c91d96c30eb4b07599b9c: done: 0.014013 sec","job_status":"done","queue_duration_s":1.766034,"scheduling_latency_s":1.766051,"redis_calls":13,"redis_duration_s":0.0014780000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000541,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000937,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009651,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014013,"completed_at":"2026-09-13T10:15:06.864Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1549gitlab # [ 265.987900] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.866Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bd4c9403168dd15e00515e66","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bd4c9403168dd15e00515e66: start","job_status":"start","queue_duration_s":1.781122,"scheduling_latency_s":1.781139}1550gitlab # [ 266.000201] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.880Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bd4c9403168dd15e00515e66","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bd4c9403168dd15e00515e66: done: 0.014223 sec","job_status":"done","queue_duration_s":1.781122,"scheduling_latency_s":1.781139,"redis_calls":13,"redis_duration_s":0.001799,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000874,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000925,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009822,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014223,"completed_at":"2026-09-13T10:15:06.880Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1551gitlab # [ 266.039227] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.881Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5bb5ad1bb3c1e05375af0c57","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5bb5ad1bb3c1e05375af0c57: start","job_status":"start","queue_duration_s":1.79652,"scheduling_latency_s":1.796538}1552gitlab # [ 266.051480] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.895Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5bb5ad1bb3c1e05375af0c57","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5bb5ad1bb3c1e05375af0c57: done: 0.014024 sec","job_status":"done","queue_duration_s":1.79652,"scheduling_latency_s":1.796538,"redis_calls":13,"redis_duration_s":0.0015739999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00054,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001034,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009826,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014024,"completed_at":"2026-09-13T10:15:06.895Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1553gitlab # [ 266.089731] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.896Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"700cdd0518a0400bf571aa71","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-700cdd0518a0400bf571aa71: start","job_status":"start","queue_duration_s":1.811713,"scheduling_latency_s":1.81173}1554gitlab # [ 266.101875] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.910Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"700cdd0518a0400bf571aa71","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-700cdd0518a0400bf571aa71: done: 0.014008 sec","job_status":"done","queue_duration_s":1.811713,"scheduling_latency_s":1.81173,"redis_calls":13,"redis_duration_s":0.001435,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000903,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00979,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014008,"completed_at":"2026-09-13T10:15:06.910Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1555gitlab # [ 266.140473] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.911Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"839bc9947e124aaf7986ce23","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-839bc9947e124aaf7986ce23: start","job_status":"start","queue_duration_s":1.826801,"scheduling_latency_s":1.826817}1556gitlab # [ 266.152609] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.925Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"839bc9947e124aaf7986ce23","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-839bc9947e124aaf7986ce23: done: 0.014279 sec","job_status":"done","queue_duration_s":1.826801,"scheduling_latency_s":1.826817,"redis_calls":13,"redis_duration_s":0.001467,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000931,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009828,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014279,"completed_at":"2026-09-13T10:15:06.925Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1557gitlab # [ 266.191465] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.927Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"577c6d09e13cccff98c51923","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-577c6d09e13cccff98c51923: start","job_status":"start","queue_duration_s":1.84216,"scheduling_latency_s":1.842176}1558gitlab # [ 266.203948] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.941Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"577c6d09e13cccff98c51923","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-577c6d09e13cccff98c51923: done: 0.014214 sec","job_status":"done","queue_duration_s":1.84216,"scheduling_latency_s":1.842176,"redis_calls":13,"redis_duration_s":0.0014810000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000538,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000943,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009789,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014214,"completed_at":"2026-09-13T10:15:06.941Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1559gitlab # [ 266.244136] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:06.942Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"edcaf123c497877f65268937","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-edcaf123c497877f65268937: start","job_status":"start","queue_duration_s":1.857506,"scheduling_latency_s":1.857523}1560gitlab # [ 266.256113] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.025Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"edcaf123c497877f65268937","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-edcaf123c497877f65268937: done: 0.082774 sec","job_status":"done","queue_duration_s":1.857506,"scheduling_latency_s":1.857523,"redis_calls":13,"redis_duration_s":0.001437,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000911,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.077681,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.082774,"completed_at":"2026-09-13T10:15:07.025Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1561gitlab # [ 266.295223] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.026Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c3d2e1fb258302f1a66dca60","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c3d2e1fb258302f1a66dca60: start","job_status":"start","queue_duration_s":1.941601,"scheduling_latency_s":1.941622}1562gitlab # [ 266.307696] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.042Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c3d2e1fb258302f1a66dca60","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c3d2e1fb258302f1a66dca60: done: 0.016352 sec","job_status":"done","queue_duration_s":1.941601,"scheduling_latency_s":1.941622,"redis_calls":13,"redis_duration_s":0.001666,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000644,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001022,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011767,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016352,"completed_at":"2026-09-13T10:15:07.042Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1563gitlab # [ 266.346403] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.043Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e7a2e9c7d8b40adc1812b946","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e7a2e9c7d8b40adc1812b946: start","job_status":"start","queue_duration_s":1.959069,"scheduling_latency_s":1.959085}1564gitlab # [ 266.358840] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.057Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e7a2e9c7d8b40adc1812b946","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e7a2e9c7d8b40adc1812b946: done: 0.014065 sec","job_status":"done","queue_duration_s":1.959069,"scheduling_latency_s":1.959085,"redis_calls":13,"redis_duration_s":0.0015119999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000552,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00096,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009616,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014065,"completed_at":"2026-09-13T10:15:07.057Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1565gitlab # [ 266.397939] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.059Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0ddffebbe6f4a53fbead6d7a","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0ddffebbe6f4a53fbead6d7a: start","job_status":"start","queue_duration_s":1.974223,"scheduling_latency_s":1.974239}1566gitlab # [ 266.410554] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.075Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0ddffebbe6f4a53fbead6d7a","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0ddffebbe6f4a53fbead6d7a: done: 0.015976 sec","job_status":"done","queue_duration_s":1.974223,"scheduling_latency_s":1.974239,"redis_calls":13,"redis_duration_s":0.001598,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000584,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001014,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009858,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015976,"completed_at":"2026-09-13T10:15:07.075Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1567gitlab # [ 266.451918] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.076Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"500bc03699ca2d3f0796b967","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-500bc03699ca2d3f0796b967: start","job_status":"start","queue_duration_s":1.991458,"scheduling_latency_s":1.991476}1568gitlab # [ 266.464477] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.093Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"500bc03699ca2d3f0796b967","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-500bc03699ca2d3f0796b967: done: 0.016731 sec","job_status":"done","queue_duration_s":1.991458,"scheduling_latency_s":1.991476,"redis_calls":13,"redis_duration_s":0.002173,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000956,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001217,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011618,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016731,"completed_at":"2026-09-13T10:15:07.092Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1569gitlab # [ 266.503295] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.094Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"57d7b81c984f893315adff85","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-57d7b81c984f893315adff85: start","job_status":"start","queue_duration_s":2.009411,"scheduling_latency_s":2.009429}1570gitlab # [ 266.515689] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.110Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"57d7b81c984f893315adff85","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-57d7b81c984f893315adff85: done: 0.01643 sec","job_status":"done","queue_duration_s":2.009411,"scheduling_latency_s":2.009429,"redis_calls":13,"redis_duration_s":0.001894,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000758,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001136,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011532,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01643,"completed_at":"2026-09-13T10:15:07.110Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1571gitlab # [ 266.554130] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.111Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"83d4f5633e3279fbcca32d67","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-83d4f5633e3279fbcca32d67: start","job_status":"start","queue_duration_s":2.027067,"scheduling_latency_s":2.027085}1572gitlab # [ 266.566157] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.128Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"83d4f5633e3279fbcca32d67","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-83d4f5633e3279fbcca32d67: done: 0.016775 sec","job_status":"done","queue_duration_s":2.027067,"scheduling_latency_s":2.027085,"redis_calls":13,"redis_duration_s":0.001873,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000675,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001198,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01096,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016775,"completed_at":"2026-09-13T10:15:07.128Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1573gitlab # [ 266.604637] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.130Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1050c8df39ce5a03d441d890","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1050c8df39ce5a03d441d890: start","job_status":"start","queue_duration_s":2.046085,"scheduling_latency_s":2.046104}1574gitlab # [ 266.616903] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.147Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1050c8df39ce5a03d441d890","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1050c8df39ce5a03d441d890: done: 0.016317 sec","job_status":"done","queue_duration_s":2.046085,"scheduling_latency_s":2.046104,"redis_calls":13,"redis_duration_s":0.0026160000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001445,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001171,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011387,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016317,"completed_at":"2026-09-13T10:15:07.147Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1575gitlab # [ 266.656545] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.148Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"71861c01e0b0e4d1b1b2895b","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-71861c01e0b0e4d1b1b2895b: start","job_status":"start","queue_duration_s":2.063572,"scheduling_latency_s":2.063588}1576gitlab # [ 266.668703] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.163Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"71861c01e0b0e4d1b1b2895b","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-71861c01e0b0e4d1b1b2895b: done: 0.014667 sec","job_status":"done","queue_duration_s":2.063572,"scheduling_latency_s":2.063588,"redis_calls":13,"redis_duration_s":0.001532,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001005,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010387,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014667,"completed_at":"2026-09-13T10:15:07.163Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1577gitlab # [ 266.706992] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.164Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1c66579fa679dbc7e5c136e3","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1c66579fa679dbc7e5c136e3: start","job_status":"start","queue_duration_s":2.079306,"scheduling_latency_s":2.079322}1578gitlab # [ 266.719470] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.178Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1c66579fa679dbc7e5c136e3","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1c66579fa679dbc7e5c136e3: done: 0.014395 sec","job_status":"done","queue_duration_s":2.079306,"scheduling_latency_s":2.079322,"redis_calls":13,"redis_duration_s":0.001545,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000561,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000984,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010176,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014395,"completed_at":"2026-09-13T10:15:07.178Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1579gitlab # [ 266.757839] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.179Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1c9659b1248368a27cf0f36e","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1c9659b1248368a27cf0f36e: start","job_status":"start","queue_duration_s":2.094806,"scheduling_latency_s":2.094822}1580gitlab # [ 266.769800] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.194Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1c9659b1248368a27cf0f36e","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1c9659b1248368a27cf0f36e: done: 0.014853 sec","job_status":"done","queue_duration_s":2.094806,"scheduling_latency_s":2.094822,"redis_calls":13,"redis_duration_s":0.00149,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000538,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000952,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009615,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014853,"completed_at":"2026-09-13T10:15:07.194Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1581gitlab # [ 266.808100] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.196Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"53ed993c56cfbaac1cbcc14a","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-53ed993c56cfbaac1cbcc14a: start","job_status":"start","queue_duration_s":2.111508,"scheduling_latency_s":2.111525}1582gitlab # [ 266.820138] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.212Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"53ed993c56cfbaac1cbcc14a","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-53ed993c56cfbaac1cbcc14a: done: 0.016039 sec","job_status":"done","queue_duration_s":2.111508,"scheduling_latency_s":2.111525,"redis_calls":13,"redis_duration_s":0.0015049999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000579,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000926,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01171,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016039,"completed_at":"2026-09-13T10:15:07.212Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1583gitlab # [ 266.859759] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.213Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dfdff65cc31430b3e005da56","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dfdff65cc31430b3e005da56: start","job_status":"start","queue_duration_s":2.128667,"scheduling_latency_s":2.128684}1584gitlab # [ 266.872326] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.228Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dfdff65cc31430b3e005da56","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dfdff65cc31430b3e005da56: done: 0.014946 sec","job_status":"done","queue_duration_s":2.128667,"scheduling_latency_s":2.128684,"redis_calls":13,"redis_duration_s":0.002266,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000544,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001722,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010599,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014946,"completed_at":"2026-09-13T10:15:07.228Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1585gitlab # [ 266.910575] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.229Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"64e91b61bca24c50d7f5b41a","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-64e91b61bca24c50d7f5b41a: start","job_status":"start","queue_duration_s":2.144753,"scheduling_latency_s":2.144769}1586gitlab # [ 266.922866] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.244Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"64e91b61bca24c50d7f5b41a","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-64e91b61bca24c50d7f5b41a: done: 0.015142 sec","job_status":"done","queue_duration_s":2.144753,"scheduling_latency_s":2.144769,"redis_calls":13,"redis_duration_s":0.00157,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000567,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001003,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010759,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015142,"completed_at":"2026-09-13T10:15:07.244Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1587gitlab # [ 266.961222] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.246Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ccb6330e77e824664a9eeb7","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ccb6330e77e824664a9eeb7: start","job_status":"start","queue_duration_s":2.161853,"scheduling_latency_s":2.161871}1588gitlab # [ 266.973405] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.262Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ccb6330e77e824664a9eeb7","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ccb6330e77e824664a9eeb7: done: 0.016235 sec","job_status":"done","queue_duration_s":2.161853,"scheduling_latency_s":2.161871,"redis_calls":13,"redis_duration_s":0.0015699999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000566,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001004,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011465,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016235,"completed_at":"2026-09-13T10:15:07.262Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1589gitlab # [ 267.012729] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.264Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fd7c0f73dafa2b125c4c9cb6","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fd7c0f73dafa2b125c4c9cb6: start","job_status":"start","queue_duration_s":2.179247,"scheduling_latency_s":2.179263}1590gitlab # [ 267.025013] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.280Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fd7c0f73dafa2b125c4c9cb6","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fd7c0f73dafa2b125c4c9cb6: done: 0.016258 sec","job_status":"done","queue_duration_s":2.179247,"scheduling_latency_s":2.179263,"redis_calls":13,"redis_duration_s":0.0017230000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000576,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001147,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011677,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016258,"completed_at":"2026-09-13T10:15:07.280Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1591gitlab # [ 267.065324] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.281Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0ab3c19078f87c7b2959f2bf","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0ab3c19078f87c7b2959f2bf: start","job_status":"start","queue_duration_s":2.196644,"scheduling_latency_s":2.196662}1592gitlab # [ 267.077561] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.298Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0ab3c19078f87c7b2959f2bf","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0ab3c19078f87c7b2959f2bf: done: 0.016918 sec","job_status":"done","queue_duration_s":2.196644,"scheduling_latency_s":2.196662,"redis_calls":13,"redis_duration_s":0.001668,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000565,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001103,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011484,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016918,"completed_at":"2026-09-13T10:15:07.298Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1593gitlab # [ 267.116493] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.299Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"23af979af93b10abd1ba6407","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-23af979af93b10abd1ba6407: start","job_status":"start","queue_duration_s":2.214638,"scheduling_latency_s":2.214655}1594gitlab # [ 267.128872] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.314Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"23af979af93b10abd1ba6407","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-23af979af93b10abd1ba6407: done: 0.014703 sec","job_status":"done","queue_duration_s":2.214638,"scheduling_latency_s":2.214655,"redis_calls":13,"redis_duration_s":0.001428,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00052,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000908,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009477,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014703,"completed_at":"2026-09-13T10:15:07.314Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1595gitlab # [ 267.167776] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.316Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be68992f72d4bfdd70040f01","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be68992f72d4bfdd70040f01: start","job_status":"start","queue_duration_s":2.231211,"scheduling_latency_s":2.231227}1596gitlab # [ 267.180345] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.331Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be68992f72d4bfdd70040f01","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be68992f72d4bfdd70040f01: done: 0.01561 sec","job_status":"done","queue_duration_s":2.231211,"scheduling_latency_s":2.231227,"redis_calls":13,"redis_duration_s":0.001505,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000572,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000933,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011168,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01561,"completed_at":"2026-09-13T10:15:07.331Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1597gitlab # [ 267.219234] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.332Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c116ab00e66824bb71300055","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c116ab00e66824bb71300055: start","job_status":"start","queue_duration_s":2.247903,"scheduling_latency_s":2.24792}1598gitlab # [ 267.231515] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.347Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c116ab00e66824bb71300055","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c116ab00e66824bb71300055: done: 0.014725 sec","job_status":"done","queue_duration_s":2.247903,"scheduling_latency_s":2.24792,"redis_calls":13,"redis_duration_s":0.001547,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00057,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000977,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010484,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014725,"completed_at":"2026-09-13T10:15:07.347Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1599gitlab # [ 267.271765] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.348Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7b122aa454edd57bfa23a459","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7b122aa454edd57bfa23a459: start","job_status":"start","queue_duration_s":2.263747,"scheduling_latency_s":2.263765}1600gitlab # [ 267.284418] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.363Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7b122aa454edd57bfa23a459","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7b122aa454edd57bfa23a459: done: 0.014961 sec","job_status":"done","queue_duration_s":2.263747,"scheduling_latency_s":2.263765,"redis_calls":13,"redis_duration_s":0.0016160000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000573,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001043,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010589,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014961,"completed_at":"2026-09-13T10:15:07.363Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1601gitlab # [ 267.323515] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.364Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"87ef63bf9db082514b6671e6","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-87ef63bf9db082514b6671e6: start","job_status":"start","queue_duration_s":2.279843,"scheduling_latency_s":2.279862}1602gitlab # [ 267.336087] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.380Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"87ef63bf9db082514b6671e6","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-87ef63bf9db082514b6671e6: done: 0.015336 sec","job_status":"done","queue_duration_s":2.279843,"scheduling_latency_s":2.279862,"redis_calls":13,"redis_duration_s":0.001674,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000619,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001055,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009941,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015336,"completed_at":"2026-09-13T10:15:07.379Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1603gitlab # [ 267.375071] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.382Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ab06132656b91eb4bb1e2c0c","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ab06132656b91eb4bb1e2c0c: start","job_status":"start","queue_duration_s":2.298055,"scheduling_latency_s":2.298074}1604gitlab # [ 267.387368] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.399Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ab06132656b91eb4bb1e2c0c","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ab06132656b91eb4bb1e2c0c: done: 0.016483 sec","job_status":"done","queue_duration_s":2.298055,"scheduling_latency_s":2.298074,"redis_calls":13,"redis_duration_s":0.001788,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000641,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001147,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011476,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016483,"completed_at":"2026-09-13T10:15:07.399Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1605gitlab # [ 267.426614] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.400Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"65294ddd69d248ec24e8c5de","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-65294ddd69d248ec24e8c5de: start","job_status":"start","queue_duration_s":2.315675,"scheduling_latency_s":2.315691}1606gitlab # [ 267.439273] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.415Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"65294ddd69d248ec24e8c5de","created_at":"2026-09-13T10:15:04.966Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-b79eaba44664408a","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0ea17d438ce4c9c502c058d26c509de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.084Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-65294ddd69d248ec24e8c5de: done: 0.014842 sec","job_status":"done","queue_duration_s":2.315675,"scheduling_latency_s":2.315691,"redis_calls":13,"redis_duration_s":0.00147,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000516,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000954,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010606,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014842,"completed_at":"2026-09-13T10:15:07.415Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1607gitlab # [ 267.479842] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.416Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"2c885a19f9948941496b84d2","created_at":"2026-09-13T10:15:05.149Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-a131c247106e4edf","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d01b9fcd3b06c6b9fdc844b41d1f4250","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.168Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-2c885a19f9948941496b84d2: start","job_status":"start","queue_duration_s":2.248203,"scheduling_latency_s":2.248219}1608gitlab # [ 267.492474] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.430Z","message":"No existing merge request to be cleaned up."}1609gitlab # [ 267.494254] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.438Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"2c885a19f9948941496b84d2","created_at":"2026-09-13T10:15:05.149Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-a131c247106e4edf","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d01b9fcd3b06c6b9fdc844b41d1f4250","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.168Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-2c885a19f9948941496b84d2: done: 0.022189 sec","job_status":"done","queue_duration_s":2.248203,"scheduling_latency_s":2.248219,"redis_calls":10,"redis_duration_s":0.0013779999999999999,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000584,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000794,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015165,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022189,"completed_at":"2026-09-13T10:15:07.438Z","load_balancing_strategy":"primary","db_duration_s":0.003926,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1610gitlab # [ 267.532755] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.440Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"59799b152686f703150c2009","created_at":"2026-09-13T10:15:05.149Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-a131c247106e4edf","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d01b9fcd3b06c6b9fdc844b41d1f4250","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.168Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-59799b152686f703150c2009: start","job_status":"start","queue_duration_s":2.272427,"scheduling_latency_s":2.272444}1611gitlab # [ 267.544990] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.451Z","message":"No existing merge request to be cleaned up."}1612gitlab # [ 267.546642] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.460Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"59799b152686f703150c2009","created_at":"2026-09-13T10:15:05.149Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-a131c247106e4edf","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d01b9fcd3b06c6b9fdc844b41d1f4250","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.168Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-59799b152686f703150c2009: done: 0.02011 sec","job_status":"done","queue_duration_s":2.272427,"scheduling_latency_s":2.272444,"redis_calls":10,"redis_duration_s":0.001379,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000613,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000766,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014202,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02011,"completed_at":"2026-09-13T10:15:07.460Z","load_balancing_strategy":"primary","db_duration_s":0.002617,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1613gitlab # [ 267.584695] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.461Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a5e736d28e5b201727deb2ae","created_at":"2026-09-13T10:15:05.149Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-a131c247106e4edf","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d01b9fcd3b06c6b9fdc844b41d1f4250","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.168Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a5e736d28e5b201727deb2ae: start","job_status":"start","queue_duration_s":2.29364,"scheduling_latency_s":2.293656}1614gitlab # [ 267.597004] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.473Z","message":"No existing merge request to be cleaned up."}1615gitlab # [ 267.598644] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.481Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a5e736d28e5b201727deb2ae","created_at":"2026-09-13T10:15:05.149Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-a131c247106e4edf","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d01b9fcd3b06c6b9fdc844b41d1f4250","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.168Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a5e736d28e5b201727deb2ae: done: 0.019979 sec","job_status":"done","queue_duration_s":2.29364,"scheduling_latency_s":2.293656,"redis_calls":10,"redis_duration_s":0.002341,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000591,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00175,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014096,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019979,"completed_at":"2026-09-13T10:15:07.481Z","load_balancing_strategy":"primary","db_duration_s":0.002464,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1616gitlab # [ 267.636911] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.483Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"2702288c75df7b1ef7eb9e66","created_at":"2026-09-13T10:15:05.149Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-a131c247106e4edf","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d01b9fcd3b06c6b9fdc844b41d1f4250","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.168Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-2702288c75df7b1ef7eb9e66: start","job_status":"start","queue_duration_s":2.314782,"scheduling_latency_s":2.314798}1617gitlab # [ 267.649294] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.495Z","message":"No existing merge request to be cleaned up."}1618gitlab # [ 267.650941] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:15:07.505Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"2702288c75df7b1ef7eb9e66","created_at":"2026-09-13T10:15:05.149Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-a131c247106e4edf","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d01b9fcd3b06c6b9fdc844b41d1f4250","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:15:05.168Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-2702288c75df7b1ef7eb9e66: done: 0.022355 sec","job_status":"done","queue_duration_s":2.314782,"scheduling_latency_s":2.314798,"redis_calls":10,"redis_duration_s":0.001608,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000656,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000952,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013786,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022355,"completed_at":"2026-09-13T10:15:07.505Z","load_balancing_strategy":"primary","db_duration_s":0.002705,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1619gitlab # [ 275.184603] setup-gitlab-start[3034]: W, [2026-09-13T10:15:17.092356 #3034] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1620gitlab # [ 309.166798] redis[841]: 10 changes in 300 seconds. Saving...1621gitlab # [ 309.173231] redis[841]: Background saving started by pid 33631622gitlab # [ 309.179537] redis[3363]: BGSAVE done, 483 keys saved, 0 keys skipped, 123054 bytes written.1623gitlab # [ 309.187861] redis[3363]: DB saved on disk1624gitlab # [ 309.190234] redis[3363]: Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB1625gitlab # [ 309.268633] redis[841]: Background saving terminated with success1626gitlab # [ 310.744346] postgres[922]: [922] LOG: checkpoint starting: time1627gitlab # [ 312.091966] gitaly[966]: time="2026-09-13T10:15:53.999Z" level=info msg="burden monitor stats" pid=9661628gitlab # [ 316.647050] postgres[3377]: [3377] ERROR: relation "table_without_models" does not exist1629gitlab # [ 316.647409] postgres[3377]: [3377] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1630gitlab # [ 319.625945] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:16:01.533Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"d4d44313fa0d8b5d86a7a401","created_at":"2026-09-13T10:16:01.461Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d9ffd91cc6437e9ec7c97a85551a7d5b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:01.528Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-d4d44313fa0d8b5d86a7a401: start","job_status":"start","queue_duration_s":0.004491,"scheduling_latency_s":0.004546}1631gitlab # [ 319.703099] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:16:01.611Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1632gitlab # [ 319.775335] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:16:01.683Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"d4d44313fa0d8b5d86a7a401","created_at":"2026-09-13T10:16:01.461Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d9ffd91cc6437e9ec7c97a85551a7d5b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:01.528Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-d4d44313fa0d8b5d86a7a401: done: 0.150147 sec","job_status":"done","queue_duration_s":0.004491,"scheduling_latency_s":0.004546,"redis_calls":15,"redis_duration_s":0.063859,"redis_read_bytes":905,"redis_write_bytes":1335,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.023618,"redis_feature_flag_read_bytes":900,"redis_feature_flag_write_bytes":514,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.036908,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003333,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038251,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.150147,"completed_at":"2026-09-13T10:16:01.683Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1633gitlab # [ 319.866599] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:16:01.746Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"2e7d4ff52a5b9132af48256a","created_at":"2026-09-13T10:16:01.545Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"be6dac4629117b60a4213ef9df3ffd4b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-13T10:16:01.554Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-2e7d4ff52a5b9132af48256a: start","job_status":"start","queue_duration_s":0.191819,"scheduling_latency_s":0.191865}1634gitlab # [ 319.998390] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:16:01.906Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"2e7d4ff52a5b9132af48256a","created_at":"2026-09-13T10:16:01.545Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"be6dac4629117b60a4213ef9df3ffd4b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-13T10:16:01.554Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-2e7d4ff52a5b9132af48256a: done: 0.159537 sec","job_status":"done","queue_duration_s":0.191819,"scheduling_latency_s":0.191865,"redis_calls":9,"redis_duration_s":0.005911,"redis_read_bytes":261,"redis_write_bytes":906,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000356,"redis_feature_flag_read_bytes":256,"redis_feature_flag_write_bytes":165,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004611,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000944,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.068,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.068,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030074,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.159537,"completed_at":"2026-09-13T10:16:01.906Z","load_balancing_strategy":"primary","db_duration_s":0.068357,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1635gitlab # [ 320.083511] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:16:01.911Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"ab6e33dc5fb7e32f3d3109bc","created_at":"2026-09-13T10:16:01.591Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"112f31cc3043a18fcd2e49101f717ddc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:01.607Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-ab6e33dc5fb7e32f3d3109bc: start","job_status":"start","queue_duration_s":0.303424,"scheduling_latency_s":0.303474}1636gitlab # [ 320.101455] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:16:01.943Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1637gitlab # [ 320.105135] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:16:01.971Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"ab6e33dc5fb7e32f3d3109bc","created_at":"2026-09-13T10:16:01.591Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"112f31cc3043a18fcd2e49101f717ddc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:01.607Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-ab6e33dc5fb7e32f3d3109bc: done: 0.060793 sec","job_status":"done","queue_duration_s":0.303424,"scheduling_latency_s":0.303474,"redis_calls":8,"redis_duration_s":0.020113,"redis_read_bytes":133,"redis_write_bytes":937,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.003697,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":111,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.013973,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002443,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020921,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.060793,"completed_at":"2026-09-13T10:16:01.971Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1638gitlab # [ 320.148760] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:16:02.018Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"295975212ecbb79e66dcc9cb","created_at":"2026-09-13T10:16:01.648Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a1ba425e363c81424afa6caa48968e98","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-13T10:16:01.660Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-295975212ecbb79e66dcc9cb: start","job_status":"start","queue_duration_s":0.357481,"scheduling_latency_s":0.357535}1639gitlab # [ 320.207112] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:16:02.115Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1640gitlab # [ 320.221319] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:16:02.129Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"295975212ecbb79e66dcc9cb","created_at":"2026-09-13T10:16:01.648Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a1ba425e363c81424afa6caa48968e98","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-13T10:16:01.660Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-295975212ecbb79e66dcc9cb: done: 0.110921 sec","job_status":"done","queue_duration_s":0.357481,"scheduling_latency_s":0.357535,"redis_calls":8,"redis_duration_s":0.002894,"redis_read_bytes":133,"redis_write_bytes":914,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000274,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":108,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001984,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000636,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023682,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.110921,"completed_at":"2026-09-13T10:16:02.129Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1641gitlab # [ 320.279441] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:16:02.132Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"c9ce20097b1dab73dcbbcca1","created_at":"2026-09-13T10:16:01.676Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7d3dc3848f1e99ce5ebece07dd673ebc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:01.688Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-c9ce20097b1dab73dcbbcca1: start","job_status":"start","queue_duration_s":0.444484,"scheduling_latency_s":0.444533}1642gitlab # [ 320.295330] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:16:02.168Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"c9ce20097b1dab73dcbbcca1","created_at":"2026-09-13T10:16:01.676Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7d3dc3848f1e99ce5ebece07dd673ebc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:01.688Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-c9ce20097b1dab73dcbbcca1: done: 0.036333 sec","job_status":"done","queue_duration_s":0.444484,"scheduling_latency_s":0.444533,"redis_calls":9,"redis_duration_s":0.003444,"redis_read_bytes":261,"redis_write_bytes":944,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000691,"redis_feature_flag_read_bytes":256,"redis_feature_flag_write_bytes":168,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002045,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000708,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024693,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036333,"completed_at":"2026-09-13T10:16:02.168Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1643gitlab # [ 320.338504] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:16:02.171Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"b4a77b85c8666cd0f271f7a7","created_at":"2026-09-13T10:16:01.704Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:01.711Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-b4a77b85c8666cd0f271f7a7: start","job_status":"start","queue_duration_s":0.460049,"scheduling_latency_s":0.460089}1644gitlab # [ 320.429642] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:16:02.338Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"b4a77b85c8666cd0f271f7a7","created_at":"2026-09-13T10:16:01.704Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:01.711Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-b4a77b85c8666cd0f271f7a7: done: 0.166497 sec","job_status":"done","queue_duration_s":0.460049,"scheduling_latency_s":0.460089,"redis_calls":65,"redis_duration_s":0.019376,"redis_read_bytes":748,"redis_write_bytes":41452,"redis_feature_flag_calls":5,"redis_feature_flag_duration_s":0.001335,"redis_feature_flag_read_bytes":640,"redis_feature_flag_write_bytes":393,"redis_queues_calls":2,"redis_queues_duration_s":0.00083,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.016156,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001055,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.149181,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.166497,"completed_at":"2026-09-13T10:16:02.337Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1645gitlab # [ 320.507978] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:16:02.341Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"9f6de28ee3ca57b292e4a15d","created_at":"2026-09-13T10:16:01.745Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8579cd00cd8da0c4b0cd2058d6e29b28","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:01.752Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-9f6de28ee3ca57b292e4a15d: start","job_status":"start","queue_duration_s":0.589101,"scheduling_latency_s":0.589149}1646gitlab # [ 320.523039] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:16:02.408Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"9f6de28ee3ca57b292e4a15d","created_at":"2026-09-13T10:16:01.745Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8579cd00cd8da0c4b0cd2058d6e29b28","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:01.752Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-9f6de28ee3ca57b292e4a15d: done: 0.066798 sec","job_status":"done","queue_duration_s":0.589101,"scheduling_latency_s":0.589149,"redis_calls":17,"redis_duration_s":0.006542,"redis_read_bytes":400,"redis_write_bytes":4270,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001035,"redis_feature_flag_read_bytes":384,"redis_feature_flag_write_bytes":248,"redis_queues_calls":2,"redis_queues_duration_s":0.000661,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3011,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.003572,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001274,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.052519,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.066798,"completed_at":"2026-09-13T10:16:02.408Z","load_balancing_strategy":"primary","db_duration_s":0.001338,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1647gitlab # [ 320.568550] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:16:02.412Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"c64eeecbb5b46f8b7b56c75b","created_at":"2026-09-13T10:16:01.756Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a52a69d1eeb392750ca053e23daec665","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:01.761Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-c64eeecbb5b46f8b7b56c75b: start","job_status":"start","queue_duration_s":0.650648,"scheduling_latency_s":0.65069}1648gitlab # [ 322.465414] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:16:04.372Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"c64eeecbb5b46f8b7b56c75b","created_at":"2026-09-13T10:16:01.756Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a52a69d1eeb392750ca053e23daec665","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:01.761Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-c64eeecbb5b46f8b7b56c75b: done: 1.959881 sec","job_status":"done","queue_duration_s":0.650648,"scheduling_latency_s":0.65069,"redis_calls":563,"redis_duration_s":1.1325370000000001,"redis_read_bytes":688,"redis_write_bytes":48843,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000326,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":83,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005557,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":1.126654,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.959779,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":1.959881,"completed_at":"2026-09-13T10:16:04.372Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1649gitlab # [ 322.477007] runuser[3033]: pam_unix(runuser:session): session closed for user gitlab1650gitlab # [ 322.563160] setup-gitlab-start[3006]: + systemctl restart gitlab-sidekiq.service1651gitlab # [ 322.564951] systemd[1]: Stopping gitlab-sidekiq.service...1652gitlab # [ 322.565991] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:16:04.389Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"9b1d172281ae974b947d2640","created_at":"2026-09-13T10:16:01.765Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"981dbef68f5a0f9f944479937809f685","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-13T10:16:01.770Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-9b1d172281ae974b947d2640: start","job_status":"start","queue_duration_s":2.618557,"scheduling_latency_s":2.618627}1653gitlab # [ 322.591241] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:16:04.499Z","message":"Shutting down"}1654gitlab # [ 322.595920] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:16:04.504Z","message":"Scheduler exiting..."}1655gitlab # [ 322.599051] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:16:04.505Z","message":"Terminating quiet threads for default capsule"}1656gitlab # [ 322.602637] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:16:04.506Z","message":"Scheduler exiting..."}1657gitlab # [ 322.688555] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:16:04.596Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1658gitlab # [ 322.790873] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:16:04.697Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"9b1d172281ae974b947d2640","created_at":"2026-09-13T10:16:01.765Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"981dbef68f5a0f9f944479937809f685","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-13T10:16:01.770Z","job_size_bytes":2,"pid":2093,"sidekiq_tid":"m7np","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-9b1d172281ae974b947d2640: done: 0.308735 sec","job_status":"done","queue_duration_s":2.618557,"scheduling_latency_s":2.618627,"redis_calls":9,"redis_duration_s":0.03721,"redis_read_bytes":7,"redis_write_bytes":1152,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.005426,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.02452,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.007264,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.070079,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.308735,"completed_at":"2026-09-13T10:16:04.697Z","load_balancing_strategy":"primary","db_duration_s":0.014533,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1659gitlab # [ 323.118928] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:16:05.027Z","message":"Bye!"}1660gitlab # [ 324.463794] sidekiq-cluster[2093]: {"severity":"INFO","time":"2026-09-13T10:16:06.371Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":2093,"worker_id":"sidekiq_0","memwd_rss_bytes":936808448,"retry":0}1661gitlab # [ 325.283970] systemd[1]: gitlab-sidekiq.service: Deactivated successfully.1662gitlab # [ 325.284398] systemd[1]: Stopped gitlab-sidekiq.service.1663gitlab # [ 325.284616] systemd[1]: gitlab-sidekiq.service: Consumed 1min 54.244s CPU time over 3min 20.428s wall clock time, 945.7M memory peak, 4.5M read from disk.1664gitlab # [ 325.298004] systemd[1]: Started gitlab-sidekiq.service.1665gitlab # [ 325.302472] setup-gitlab-start[3006]: + mkdir -p /var/lib/secrets1666gitlab # [ 325.307318] setup-gitlab-start[3006]: + echo nixbot-test-token-of-20-chars1667gitlab # [ 325.310389] setup-gitlab-start[3006]: + chmod 644 /var/lib/secrets/gitlab-token1668gitlab # [ 325.313491] setup-gitlab-start[3006]: + timeout 300 bash -c $'until curl -fs -H "PRIVATE-TOKEN: $0" http://gitlab/api/v4/projects/root%2Ftest-flake > /dev/null; do\n curl -s -X POST -H "PRIVATE-TOKEN: $0" -H "Content-Type: application/json" \\\n -d "{\\"name\\":\\"test-flake\\",\\"visibility\\":\\"public\\",\\"topics\\":[\\"build-with-buildbot\\"]}" \\\n http://gitlab/api/v4/projects | jq -c "{id, message, error}"\n sleep 5\ndone' nixbot-test-token-of-20-chars1669gitlab # [ 325.865970] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:16:07 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 404 35 "" "curl/8.21.0" 5071670gitlab # [ 327.055533] gitaly[966]: time="2026-09-13T10:16:08.963Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M2D48RBD91NEYGQF89A3QRJ6 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T10:16:18.009" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:16:08.962" pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1671gitlab # [ 327.064589] gitaly[966]: time="2026-09-13T10:16:08.964Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2D48RBD91NEYGQF89A3QRJ6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T10:16:18.009" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:16:08.962" grpc.time_ms=1.285 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1672gitlab # [ 327.335202] gitaly[966]: time="2026-09-13T10:16:09.243Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M2D48RBD91NEYGQF89A3QRJ6 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T10:16:19.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:16:09.243" pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1673gitlab # [ 327.382854] gitaly[966]: time="2026-09-13T10:16:09.244Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2D48RBD91NEYGQF89A3QRJ6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T10:16:19.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:16:09.243" grpc.time_ms=0.667 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1674gitlab # [ 327.422176] gitaly[966]: time="2026-09-13T10:16:09.331Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2D48RBD91NEYGQF89A3QRJ6 diskcache=e7680b98-2ae6-4921-907e-b1bd91f98b2d grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-13T10:16:19.022" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:16:09.263" pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1675gitlab # [ 327.443089] gitaly[966]: time="2026-09-13T10:16:09.331Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=37 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1439 command.oublock=32 command.real_time_ms=43 command.spawn_token_fork_ms=9 command.system_time_ms=31 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M2D48RBD91NEYGQF89A3QRJ6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-13T10:16:19.022" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:16:09.263" grpc.time_ms=68.131 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1676gitlab # [ 327.465784] gitaly[966]: time="2026-09-13T10:16:09.355Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M2D48RBD91NEYGQF89A3QRJ6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T10:16:19.113" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:16:09.353" pid=966 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1677gitlab # [ 327.480552] gitaly[966]: time="2026-09-13T10:16:09.355Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2D48RBD91NEYGQF89A3QRJ6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T10:16:19.113" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:16:09.353" grpc.time_ms=1.68 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1678gitlab # [ 327.494079] gitaly[966]: time="2026-09-13T10:16:09.382Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D48RBD91NEYGQF89A3QRJ6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-13T10:16:19.124" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:16:09.364" grpc.time_ms=17.229 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1679gitlab # [ 327.596929] gitaly[966]: time="2026-09-13T10:16:09.505Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M2D48RBD91NEYGQF89A3QRJ6 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T10:16:19.264" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:16:09.505" pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1680gitlab # [ 327.611748] gitaly[966]: time="2026-09-13T10:16:09.506Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2D48RBD91NEYGQF89A3QRJ6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T10:16:19.264" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:16:09.505" grpc.time_ms=0.586 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1681gitlab # [ 327.662262] gitaly[966]: time="2026-09-13T10:16:09.571Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2D48RBD91NEYGQF89A3QRJ6 diskcache=33009767-b7e0-4336-b01e-9d2fb17f33cd grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-13T10:16:19.272" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:16:09.513" pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1682gitlab # [ 327.679719] gitaly[966]: time="2026-09-13T10:16:09.571Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=29 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1439 command.oublock=32 command.real_time_ms=33 command.spawn_token_fork_ms=5 command.system_time_ms=20 command.user_time_ms=9 component=gitaly.UnaryServerInterceptor correlation_id=01M2D48RBD91NEYGQF89A3QRJ6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-13T10:16:19.272" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:16:09.513" grpc.time_ms=57.994 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1683gitlab # [ 327.699978] gitaly[966]: time="2026-09-13T10:16:09.584Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M2D48RBD91NEYGQF89A3QRJ6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T10:16:19.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:16:09.582" pid=966 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1684gitlab # [ 327.713231] gitaly[966]: time="2026-09-13T10:16:09.584Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2D48RBD91NEYGQF89A3QRJ6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T10:16:19.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:16:09.582" grpc.time_ms=1.464 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1685gitlab # [ 327.727044] gitaly[966]: time="2026-09-13T10:16:09.608Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D48RBD91NEYGQF89A3QRJ6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-13T10:16:19.011" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:16:09.591" grpc.time_ms=16.295 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1686gitlab # [ 327.759286] gitaly[966]: time="2026-09-13T10:16:09.668Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2D48RBD91NEYGQF89A3QRJ6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T10:16:19.087" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:16:09.667" grpc.time_ms=0.333 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1687gitlab # [ 327.775241] gitaly[966]: time="2026-09-13T10:16:09.682Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2D48RBD91NEYGQF89A3QRJ6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-13T10:16:19.101" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:16:09.681" grpc.time_ms=0.519 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1688gitlab # [ 328.612099] gitaly[966]: time="2026-09-13T10:16:10.519Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2D48RBD91NEYGQF89A3QRJ6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-13T10:16:20.265" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:16:10.506" grpc.time_ms=13.559 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1689gitlab # [ 329.071633] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:16:10 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 31891690gitlab # [ 329.078151] setup-gitlab-start[3447]: {"id":1,"message":null,"error":null}1691gitlab # [ 330.438989] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T10:16:12.318Z","message":"Starting cluster with 1 processes"}1692gitlab # [ 334.759493] gitaly[966]: time="2026-09-13T10:16:16.667Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2D490CH3ZSETXWRA2YQZQRF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-13T10:16:26.008" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:16:16.649" grpc.time_ms=18.068 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1693gitlab # [ 335.613593] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:16:17 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 200 4924 "" "curl/8.21.0" 15041694gitlab # [ 335.623649] setup-gitlab-start[3006]: + mkdir -p /tmp/test-flake1695gitlab # [ 335.628846] setup-gitlab-start[3006]: + cd /tmp/test-flake1696gitlab # [ 335.630054] setup-gitlab-start[3006]: + git init -b master1697gitlab # [ 335.660786] setup-gitlab-start[3543]: Initialized empty Git repository in /tmp/test-flake/.git/1698gitlab # [ 335.663333] setup-gitlab-start[3006]: + git config user.name test1699gitlab # [ 335.668942] setup-gitlab-start[3006]: + git config user.email test@example.com1700gitlab # [ 335.675825] setup-gitlab-start[3006]: + cat1701gitlab # [ 335.682062] setup-gitlab-start[3006]: + git add flake.nix1702gitlab # [ 335.691470] setup-gitlab-start[3006]: + git commit -m 'initial commit'1703gitlab # [ 335.720837] setup-gitlab-start[3548]: [master (root-commit) a97336d] initial commit1704gitlab # [ 335.724060] setup-gitlab-start[3548]: 1 file changed, 10 insertions(+)1705gitlab # [ 335.726709] setup-gitlab-start[3548]: create mode 100644 flake.nix1706gitlab # [ 335.729464] setup-gitlab-start[3006]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git1707gitlab # [ 335.734192] setup-gitlab-start[3006]: + git push -u origin master1708gitlab # [ 335.850478] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:16:17 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 711709gitlab # [ 336.018077] gitaly[966]: time="2026-09-13T10:16:17.925Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2D49231XJS3EVX07Y98GMH7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:16:17.908" grpc.time_ms=17.296 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1710gitlab # [ 336.056376] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:16:17 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 1661711gitlab # [ 336.511426] gitaly[966]: time="2026-09-13T10:16:18.420Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4929ZQRKFKW6TJVV0RNR0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-13T10:16:28.002" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:16:18.402" grpc.time_ms=17.248 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1712gitlab # [ 336.583667] gitaly[966]: time="2026-09-13T10:16:18.492Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=301 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D4929ZQRKFKW6TJVV0RNR0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-13T10:16:48.073" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-13T10:16:18.473" grpc.time_ms=18.697 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1713gitlab # [ 336.655699] gitaly[966]: time="2026-09-13T10:16:18.563Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M2D4929ZQRKFKW6TJVV0RNR0 duration_ms=297ns method=POST pid=966 status=200 url="http://unix/api/v4/internal/allowed"1714gitlab # [ 336.732262] gitaly[966]: time="2026-09-13T10:16:18.641Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M2D4929ZQRKFKW6TJVV0RNR0 duration_ms=76ns method=POST pid=966 status=200 url="http://unix/api/v4/internal/pre_receive"1715gitlab # [ 336.741672] gitaly[966]: time="2026-09-13T10:16:18.650Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D4929ZQRKFKW6TJVV0RNR0 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5316 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:16:18.263" grpc.time_ms=386.583 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1716gitlab # [ 336.829535] gitaly[966]: time="2026-09-13T10:16:18.738Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D4929ZQRKFKW6TJVV0RNR0 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4786 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:16:18.736" grpc.time_ms=1.547 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1717gitlab # [ 336.891964] gitaly[966]: time="2026-09-13T10:16:18.800Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D4929ZQRKFKW6TJVV0RNR0 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4785 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:16:18.799" grpc.time_ms=1.144 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1718gitlab # [ 336.975921] gitaly[966]: time="2026-09-13T10:16:18.884Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D4929ZQRKFKW6TJVV0RNR0 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:16:18.880" grpc.time_ms=4.217 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1719gitlab # [ 337.204987] gitaly[966]: time="2026-09-13T10:16:19.112Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4929ZQRKFKW6TJVV0RNR0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-13T10:16:29.004" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:16:19.094" grpc.time_ms=17.721 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1720gitlab # [ 337.298006] gitaly[966]: time="2026-09-13T10:16:19.206Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=32 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1433 command.oublock=0 command.real_time_ms=34 command.spawn_token_fork_ms=5 command.system_time_ms=32 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4929ZQRKFKW6TJVV0RNR0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-13T10:16:29.074" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:16:19.165" grpc.time_ms=41.008 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1721gitlab # [ 337.358310] gitaly[966]: time="2026-09-13T10:16:19.267Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M2D4929ZQRKFKW6TJVV0RNR0 duration_ms=331ns method=POST pid=966 status=200 url="http://unix/api/v4/internal/post_receive"1722gitlab # [ 337.367205] gitaly[966]: time="2026-09-13T10:16:19.268Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D4929ZQRKFKW6TJVV0RNR0 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4810 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:16:18.934" grpc.time_ms=334.23 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1723gitlab # [ 337.398372] gitaly[966]: time="2026-09-13T10:16:19.285Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M2D4929ZQRKFKW6TJVV0RNR0 diskcache=035f1c28-7c5b-4a76-8d63-910c80a9ae7e grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:16:18.148" pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1724gitlab # [ 337.424376] systemd[1]: Finished setup-gitlab.service.1725gitlab # [ 337.426631] gitlab-workhorse[885]: time="2026-09-13T10:16:19Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521726gitlab # [ 337.430277] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:16:19 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 13021727gitlab # [ 337.433977] setup-gitlab-start[3552]: To http://gitlab/root/test-flake.git1728gitlab # [ 337.435560] setup-gitlab-start[3552]: * [new branch] master -> master1729gitlab # [ 337.437187] setup-gitlab-start[3552]: branch 'master' set up to track 'origin/master'.1730gitlab # [ 337.438939] gitaly[966]: time="2026-09-13T10:16:19.286Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=370 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=847872 command.minflt=13422 command.oublock=32 command.real_time_ms=1126 command.spawn_token_fork_ms=2 command.system_time_ms=183 command.user_time_ms=187 component=gitaly.StreamServerInterceptor correlation_id=01M2D4929ZQRKFKW6TJVV0RNR0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:16:18.148" grpc.time_ms=1137.441 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1731gitlab # [ 337.458736] systemd[1]: Starting nixbot CI service...1732gitlab: (finished: waiting for unit setup-gitlab.service, in 338.41 seconds)1733gitlab: waiting for unit nixbot.service1734gitlab # [ 338.747935] nixbot[3628]: {"ts": "2026-09-13T10:16:20.655Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1735gitlab # [ 338.790060] nixbot[3628]: {"ts": "2026-09-13T10:16:20.698Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1736gitlab # [ 338.829630] nixbot[3628]: {"ts": "2026-09-13T10:16:20.738Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1737gitlab # [ 338.837147] nixbot[3628]: {"ts": "2026-09-13T10:16:20.746Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1738gitlab # [ 338.844086] nixbot[3628]: {"ts": "2026-09-13T10:16:20.751Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1739gitlab # [ 338.852899] nixbot[3628]: {"ts": "2026-09-13T10:16:20.761Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1740gitlab # [ 338.859475] nixbot[3628]: {"ts": "2026-09-13T10:16:20.768Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1741gitlab # [ 338.866131] nixbot[3628]: {"ts": "2026-09-13T10:16:20.771Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1742gitlab # [ 338.872782] nixbot[3628]: {"ts": "2026-09-13T10:16:20.781Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1743gitlab # [ 338.879257] nixbot[3628]: {"ts": "2026-09-13T10:16:20.785Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1744gitlab # [ 338.885989] nixbot[3628]: {"ts": "2026-09-13T10:16:20.794Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1745gitlab # [ 338.898032] nixbot[3628]: {"ts": "2026-09-13T10:16:20.806Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1746gitlab # [ 338.904647] nixbot[3628]: {"ts": "2026-09-13T10:16:20.813Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1747gitlab # [ 338.911329] nixbot[3628]: {"ts": "2026-09-13T10:16:20.816Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1748gitlab # [ 338.917738] nixbot[3628]: {"ts": "2026-09-13T10:16:20.824Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1749gitlab # [ 338.924057] nixbot[3628]: {"ts": "2026-09-13T10:16:20.827Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1750gitlab # [ 338.929777] nixbot[3628]: {"ts": "2026-09-13T10:16:20.830Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1751gitlab # [ 338.935455] nixbot[3628]: {"ts": "2026-09-13T10:16:20.833Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1752gitlab # [ 338.940786] nixbot[3628]: {"ts": "2026-09-13T10:16:20.841Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"}1753gitlab # [ 338.946081] nixbot[3628]: {"ts": "2026-09-13T10:16:20.844Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"}1754gitlab # [ 338.951292] nixbot[3628]: {"ts": "2026-09-13T10:16:20.847Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"}1755gitlab # [ 338.956299] nixbot[3628]: {"ts": "2026-09-13T10:16:20.857Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"}1756gitlab # [ 338.961321] nixbot[3628]: {"ts": "2026-09-13T10:16:20.860Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"}1757gitlab # [ 338.967718] nixbot[3628]: {"ts": "2026-09-13T10:16:20.863Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"}1758gitlab # [ 338.972543] nixbot[3628]: {"ts": "2026-09-13T10:16:20.865Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"}1759gitlab # [ 338.976910] nixbot[3628]: {"ts": "2026-09-13T10:16:20.869Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"}1760gitlab # [ 338.981217] nixbot[3628]: {"ts": "2026-09-13T10:16:20.872Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"}1761gitlab # [ 338.985169] nixbot[3628]: {"ts": "2026-09-13T10:16:20.874Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"}1762gitlab # [ 338.988974] nixbot[3628]: {"ts": "2026-09-13T10:16:20.877Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"}1763gitlab # [ 338.992649] nixbot[3628]: {"ts": "2026-09-13T10:16:20.899Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"}1764gitlab # [ 338.996765] nixbot[3628]: {"ts": "2026-09-13T10:16:20.905Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"}1765gitlab # [ 339.000599] nixbot[3628]: {"ts": "2026-09-13T10:16:20.908Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"}1766gitlab # [ 339.004399] nixbot[3628]: {"ts": "2026-09-13T10:16:20.910Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 32, "migration": "upload_queue"}1767gitlab # [ 339.008410] nixbot[3628]: {"ts": "2026-09-13T10:16:20.917Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 33, "migration": "effect_runs_catchup"}1768gitlab # [ 339.012597] nixbot[3628]: {"ts": "2026-09-13T10:16:20.921Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 35, "migration": "settle_reset_effects"}1769gitlab # [ 339.484776] nixbot[3628]: INFO: Started server process [3628]1770gitlab # [ 339.487654] nixbot[3628]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)1771gitlab # [ 339.512605] nixbot[3628]: INFO: ::ffff:127.0.0.1:41480 - "GET /health HTTP/1.1" 200 OK1772gitlab # [ 339.524321] systemd[1]: Started nixbot CI service.1773gitlab # [ 339.525940] systemd[1]: Reached target Multi-User System.1774gitlab # [ 339.528450] systemd[1]: Startup finished in 730ms (kernel) + 4.161s (initrd) + 5min 34.636s (userspace) = 5min 39.528s.1775gitlab # [ 339.974002] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:16:21 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 4601776gitlab # [ 339.979759] nixbot[3628]: {"ts": "2026-09-13T10:16:21.888Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1777gitlab # [ 340.004325] nixbot[3628]: {"ts": "2026-09-13T10:16:21.913Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1778gitlab: (finished: waiting for unit nixbot.service, in 2.15 seconds)1779gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1780gitlab # [ 340.135326] nixbot[3628]: INFO: ::ffff:127.0.0.1:41492 - "GET /health HTTP/1.1" 200 OK1781gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.04 seconds)1782subtest: project discovered and webhook registered1783gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1784gitlab # [ 340.159879] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:16:22 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 1421785gitlab # [ 340.163674] nixbot[3628]: {"ts": "2026-09-13T10:16:22.071Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1786gitlab # [ 340.167656] nixbot[3628]: {"ts": "2026-09-13T10:16:22.074Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1787gitlab # [ 340.280426] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:16:22 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.21.0" 911788gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.14 seconds)1789gitlab # [ 340.443608] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:16:22 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 918 "" "python-httpx/0.28.1" 2691790gitlab # [ 340.450435] nixbot[3628]: {"ts": "2026-09-13T10:16:22.353Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""}1791gitlab # [ 340.718934] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:16:22 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 2631792gitlab # [ 340.726497] nixbot[3628]: {"ts": "2026-09-13T10:16:22.628Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1793gitlab # [ 340.844763] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:16:22 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "python-httpx/0.28.1" 1141794gitlab # [ 340.846910] nixbot[3628]: {"ts": "2026-09-13T10:16:22.755Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1795gitlab # [ 340.992618] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:16:22 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 918 "" "python-httpx/0.28.1" 1361796gitlab # [ 341.000170] nixbot[3628]: {"ts": "2026-09-13T10:16:22.901Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""}1797gitlab # [ 341.137203] gitaly[966]: time="2026-09-13T10:16:23.044Z" level=info msg="finished unary call with code OK" catfile.duration_ms=6 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=6 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=576 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=3 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2D497407YP4NHW2ZG8TF5KW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-09-13T10:16:53.016" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:16:23.017" grpc.time_ms=26.465 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1798gitlab # [ 341.170398] gitaly[966]: time="2026-09-13T10:16:23.079Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D497407YP4NHW2ZG8TF5KW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:16:53.076" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:16:23.077" grpc.time_ms=1.264 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1799gitlab # [ 341.202580] gitaly[966]: time="2026-09-13T10:16:23.103Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=581 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D497407YP4NHW2ZG8TF5KW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-09-13T10:16:33.004" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:16:23.085" grpc.time_ms=17.848 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1800gitlab # [ 341.254527] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:16:23 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 2501801gitlab # [ 341.260751] nixbot[3628]: {"ts": "2026-09-13T10:16:23.165Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""}1802gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1803gitlab # [ 341.407066] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:16:23 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 1421804gitlab # [ 341.413015] nixbot[3628]: {"ts": "2026-09-13T10:16:23.319Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""}1805gitlab # [ 341.422895] nixbot[3628]: {"ts": "2026-09-13T10:16:23.331Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1806gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.18 seconds)1807(finished: subtest: project discovered and webhook registered, in 1.32 seconds)1808subtest: push delivers the webhook1809gitlab: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails'1810gitlab # [ 341.460189] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:16:23 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "curl/8.21.0" 1201811gitlab # [ 341.597685] gitaly[966]: time="2026-09-13T10:16:23.506Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=3 command.system_time_ms=7 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2D497KPXGET0SV4FCCT646P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:16:23.490" grpc.time_ms=16.274 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc1812gitlab # [ 341.638104] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:16:23 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 921813gitlab # [ 341.777585] gitaly[966]: time="2026-09-13T10:16:23.686Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2D497PPE53EFJV806GASHY9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:16:23.667" pid=966 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc1814gitlab # [ 341.797500] gitaly[966]: time="2026-09-13T10:16:23.686Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=589 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2D497PPE53EFJV806GASHY9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:16:23.667" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc1815gitlab # [ 341.834606] gitlab-workhorse[885]: time="2026-09-13T10:16:23Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1131816gitlab # [ 341.839630] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:16:23 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 1781817gitlab # [ 341.843939] gitaly[966]: time="2026-09-13T10:16:23.687Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=589 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2D497PPE53EFJV806GASHY9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:16:23.667" grpc.time_ms=19.283 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc1818gitlab # [ 341.880308] gitaly[966]: time="2026-09-13T10:16:23.789Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2D497WG8H6BJ5F0RH295E3Q grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:16:23.766" pid=966 remote_ip="2001:db8:1::1" request_sha=0ec5963f58ca3e0f4fd056e8efbad41c8727bc3a response_bytes=228 span.kind=server system=grpc1819gitlab # [ 341.902039] gitaly[966]: time="2026-09-13T10:16:23.789Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1135 command.oublock=0 command.real_time_ms=20 command.spawn_token_fork_ms=6 command.system_time_ms=18 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D497WG8H6BJ5F0RH295E3Q grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:16:23.766" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc1820gitlab # [ 341.928654] gitlab-workhorse[885]: time="2026-09-13T10:16:23Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2281821gitlab # [ 341.931885] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:16:23 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 228 "" "git/2.55.0" 931822gitlab # [ 341.934968] gitaly[966]: time="2026-09-13T10:16:23.790Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1135 command.oublock=0 command.real_time_ms=20 command.spawn_token_fork_ms=6 command.system_time_ms=18 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D497WG8H6BJ5F0RH295E3Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:16:23.766" grpc.time_ms=23.581 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc1823gitlab # [ 342.135081] gitaly[966]: time="2026-09-13T10:16:24.043Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2D4980YX8WKMVJVPQCDYMD8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:16:24.026" grpc.time_ms=17.455 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc1824gitlab # [ 342.174317] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:16:24 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 2051825gitlab # [ 342.216919] gitaly[966]: time="2026-09-13T10:16:24.124Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2D4987DED4CPDHHNQRC48RZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:16:24.109" pid=966 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1826gitlab # [ 342.247558] gitaly[966]: time="2026-09-13T10:16:24.124Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4987DED4CPDHHNQRC48RZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:16:24.109" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc1827gitlab # [ 342.279514] gitlab-workhorse[885]: time="2026-09-13T10:16:24Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671828gitlab # [ 342.283189] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:16:24 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 801829gitlab # [ 342.286700] gitaly[966]: time="2026-09-13T10:16:24.124Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4987DED4CPDHHNQRC48RZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:16:24.109" grpc.time_ms=15.488 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc1830gitlab # [ 342.364138] gitaly[966]: time="2026-09-13T10:16:24.272Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D498BYZSTE6KA14JKAR2QJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:16:24.258" grpc.time_ms=14.314 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc1831gitlab # [ 342.399425] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:16:24 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 871832gitlab # [ 342.524000] gitaly[966]: time="2026-09-13T10:16:24.432Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2D498ERDN4AGAM50Z61YHSZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:16:24.399" pid=966 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc1833gitlab # [ 342.551988] gitaly[966]: time="2026-09-13T10:16:24.437Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=27 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1103 command.oublock=0 command.real_time_ms=29 command.spawn_token_fork_ms=2 command.system_time_ms=23 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M2D498ERDN4AGAM50Z61YHSZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:16:24.399" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc1834gitlab # [ 342.583439] gitlab-workhorse[885]: time="2026-09-13T10:16:24Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2091835gitlab # [ 342.586769] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:16:24 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 1581836gitlab # [ 342.590084] gitaly[966]: time="2026-09-13T10:16:24.438Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=27 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1103 command.oublock=0 command.real_time_ms=29 command.spawn_token_fork_ms=2 command.system_time_ms=23 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M2D498ERDN4AGAM50Z61YHSZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:16:24.399" grpc.time_ms=38.667 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc1837gitlab # [ 342.707600] gitaly[966]: time="2026-09-13T10:16:24.616Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D498P87CGPP5YH2KQ33ZAW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:16:54.214" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:16:24.615" grpc.time_ms=0.911 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1838gitlab # [ 342.845912] gitaly[966]: time="2026-09-13T10:16:24.754Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=587 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D498P87CGPP5YH2KQ33ZAW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-13T10:16:54.334" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:16:24.735" grpc.time_ms=19.47 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1839gitlab # [ 342.938590] gitaly[966]: time="2026-09-13T10:16:24.847Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2D498P87CGPP5YH2KQ33ZAW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:16:34.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:16:24.828" grpc.time_ms=18.301 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1840gitlab # [ 343.014545] gitaly[966]: time="2026-09-13T10:16:24.923Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D498P87CGPP5YH2KQ33ZAW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:16:34.085" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:16:24.905" grpc.time_ms=17.539 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1841gitlab # [ 343.195074] gitaly[966]: time="2026-09-13T10:16:25.103Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2D498P87CGPP5YH2KQ33ZAW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-13T10:16:35.022" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:16:25.103" grpc.time_ms=0.617 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1842gitlab # [ 343.262960] gitaly[966]: time="2026-09-13T10:16:25.170Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=570 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=7 component=gitaly.UnaryServerInterceptor correlation_id=01M2D498P87CGPP5YH2KQ33ZAW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-13T10:16:35.071" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:16:25.152" grpc.time_ms=17.862 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1843gitlab # [ 343.342049] gitaly[966]: time="2026-09-13T10:16:25.250Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D498P87CGPP5YH2KQ33ZAW grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4154 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:16:25.248" grpc.time_ms=2.2 pid=966 span.kind=server system=grpc1844gitlab # [ 343.424443] gitaly[966]: time="2026-09-13T10:16:25.333Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D498P87CGPP5YH2KQ33ZAW grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4156 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:16:25.332" grpc.time_ms=0.877 pid=966 span.kind=server system=grpc1845gitlab # [ 343.453618] gitaly[966]: time="2026-09-13T10:16:25.351Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2D498P87CGPP5YH2KQ33ZAW diskcache=f9fef606-5159-465a-ad5c-ff707303bc37 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-13T10:16:35.095" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:16:25.176" pid=966 ref=refs/keep-around/a97336d9b6bb4bb908a8849d973a90d364eab2ac remote_ip="2001:db8:1::1" revision=a97336d9b6bb4bb908a8849d973a90d364eab2ac span.kind=server system=grpc username=root1846gitlab # [ 343.481995] gitaly[966]: time="2026-09-13T10:16:25.352Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=144 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5420 command.oublock=8 command.real_time_ms=167 command.spawn_token_fork_ms=2 command.system_time_ms=102 command.user_time_ms=42 component=gitaly.UnaryServerInterceptor correlation_id=01M2D498P87CGPP5YH2KQ33ZAW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-13T10:16:35.095" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:16:25.176" grpc.time_ms=175.541 pid=966 ref=refs/keep-around/a97336d9b6bb4bb908a8849d973a90d364eab2ac remote_ip="2001:db8:1::1" revision=a97336d9b6bb4bb908a8849d973a90d364eab2ac span.kind=server system=grpc username=root1847gitlab # [ 343.509929] gitaly[966]: time="2026-09-13T10:16:25.361Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D498P87CGPP5YH2KQ33ZAW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:16:55.359" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:16:25.360" grpc.time_ms=0.793 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1848gitlab # [ 343.737649] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:16:25 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/a97336d9b6bb4bb908a8849d973a90d364eab2ac HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 11251849gitlab # [ 343.743684] nixbot[3628]: {"ts": "2026-09-13T10:16:25.647Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/a97336d9b6bb4bb908a8849d973a90d364eab2ac \"HTTP/1.1 201 Created\""}1850gitlab # [ 343.757315] nixbot[3628]: {"ts": "2026-09-13T10:16:25.666Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-a97336d9b6bb-0"}1851gitlab # [ 344.009811] nixbot[3628]: {"ts": "2026-09-13T10:16:25.918Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-a97336d9b6bb-0", "argv0": "bwrap"}1852gitlab # [ 344.199963] systemd[1]: Started Nix Daemon.1853gitlab # [ 344.304979] nix-daemon[3797]: accepted connection from pid 3792, user nixbot1854gitlab # [ 349.253006] nix-daemon[3797]: accepted connection from pid 3847, user nixbot1855gitlab # [ 349.287493] nix-daemon[3797]: accepted connection from pid 3792, user nixbot1856gitlab # [ 353.997257] nix-daemon[3797]: accepted connection from pid 3792, user nixbot1857gitlab # [ 354.202769] gitaly[966]: time="2026-09-13T10:16:36.110Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D49KWYVEM8GH4B3AT2HV1P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:17:06.108" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:16:36.109" grpc.time_ms=1.194 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1858gitlab # [ 354.282756] gitaly[966]: time="2026-09-13T10:16:36.191Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2D49KWYVEM8GH4B3AT2HV1P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:16:46.093" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:16:36.173" grpc.time_ms=17.38 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1859gitlab # [ 354.367258] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:16:36 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/a97336d9b6bb4bb908a8849d973a90d364eab2ac HTTP/1.1" 201 676 "" "python-httpx/0.28.1" 2761860gitlab # [ 354.377688] nixbot[3628]: {"ts": "2026-09-13T10:16:36.277Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/a97336d9b6bb4bb908a8849d973a90d364eab2ac \"HTTP/1.1 201 Created\""}1861gitlab # [ 354.452342] gitaly[966]: time="2026-09-13T10:16:36.361Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D49M5WGYD147ZTYG1TG1J9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:17:06.358" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:16:36.359" grpc.time_ms=1.083 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1862gitlab # [ 354.526610] gitaly[966]: time="2026-09-13T10:16:36.435Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2D49M5WGYD147ZTYG1TG1J9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:16:46.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:16:36.420" grpc.time_ms=14.895 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1863gitlab # [ 354.631103] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:16:36 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/a97336d9b6bb4bb908a8849d973a90d364eab2ac HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 2551864gitlab # [ 354.639228] nixbot[3628]: {"ts": "2026-09-13T10:16:36.541Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/a97336d9b6bb4bb908a8849d973a90d364eab2ac \"HTTP/1.1 201 Created\""}1865gitlab # [ 354.780166] gitaly[966]: time="2026-09-13T10:16:36.689Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D49MEK8A25Z2YW8BXDKG3G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:17:06.086" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:16:36.687" grpc.time_ms=1.209 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1866gitlab # [ 354.942427] gitaly[966]: time="2026-09-13T10:16:36.851Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D49MEK8A25Z2YW8BXDKG3G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:16:46.252" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:16:36.833" grpc.time_ms=17.872 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1867gitlab # [ 355.116202] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:16:37 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/a97336d9b6bb4bb908a8849d973a90d364eab2ac HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 4611868gitlab # [ 355.124414] nixbot[3628]: {"ts": "2026-09-13T10:16:37.026Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/a97336d9b6bb4bb908a8849d973a90d364eab2ac \"HTTP/1.1 201 Created\""}1869gitlab # [ 355.750988] gitaly[966]: time="2026-09-13T10:16:37.658Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2D49ND6PQCWRM85TV5C2W8D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=192 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:16:37.640" grpc.time_ms=18.011 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc1870gitlab # [ 355.777021] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:16:37 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1161871gitlab # [ 355.878636] gitaly[966]: time="2026-09-13T10:16:37.787Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2D49NGY42YZTFH3B88K7AMS grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:16:37.768" pid=966 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1872gitlab # [ 355.909228] gitaly[966]: time="2026-09-13T10:16:37.787Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M2D49NGY42YZTFH3B88K7AMS grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:16:37.768" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc1873gitlab # [ 355.936498] gitlab-workhorse[885]: time="2026-09-13T10:16:37Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671874gitlab # [ 355.939580] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:16:37 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1261875gitlab # [ 355.942841] gitaly[966]: time="2026-09-13T10:16:37.788Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M2D49NGY42YZTFH3B88K7AMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:16:37.768" grpc.time_ms=19.764 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc1876gitlab # [ 356.059044] gitaly[966]: time="2026-09-13T10:16:37.967Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=4 command.system_time_ms=0 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01M2D49NR28CKA5R79EP905WVR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:16:37.955" grpc.time_ms=11.89 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc1877gitlab # [ 356.075755] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:16:37 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 781878gitlab # [ 356.150338] gitaly[966]: time="2026-09-13T10:16:38.059Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2D49NTKY347HGX1F3CA4TSV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:16:38.042" pid=966 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc1879gitlab # [ 356.163621] gitaly[966]: time="2026-09-13T10:16:38.059Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2D49NTKY347HGX1F3CA4TSV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:16:38.042" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc1880gitlab # [ 356.178678] gitlab-workhorse[885]: time="2026-09-13T10:16:38Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671881gitlab # [ 356.180704] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:16:38 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 881882gitlab # [ 356.182656] gitaly[966]: time="2026-09-13T10:16:38.059Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2D49NTKY347HGX1F3CA4TSV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:16:38.042" grpc.time_ms=17.496 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc1883gitlab # [ 371.039476] sidekiq-cluster[3504]: 2026-09-13T10:16:52.946Z pid=3504 tid=yl4 INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>3504, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>428666880}1884gitlab # [ 371.331312] sidekiq-cluster[3504]: W, [2026-09-13T10:16:53.240081 #3504] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1885gitlab # [ 371.397755] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:16:53.306Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1886gitlab # [ 374.927476] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:16:56.835Z","message":"GitLab reliable fetch activated!"}1887gitlab # [ 374.927854] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:16:56.835Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1888gitlab # [ 412.684398] postgres[4358]: [4358] ERROR: relation "table_without_models" does not exist1889gitlab # [ 412.684974] postgres[4358]: [4358] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1890gitlab # [ 417.525315] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:39.431Z","message":"Booted Rails 7.2.3.1 application in production environment"}1891gitlab # [ 417.525816] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:39.433Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1892gitlab # [ 417.525997] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:39.433Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1893gitlab # [ 417.526118] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:39.433Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1894gitlab # [ 417.557170] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:39.435Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1895gitlab: (finished: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails', in 76.52 seconds)1896gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1897gitlab # [ 418.206107] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:17:40 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 1291898gitlab # [ 418.326851] gitaly[966]: time="2026-09-13T10:17:40.234Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D4BJGKAXP773YNVPXR5DHF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=246 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:17:40.218" grpc.time_ms=16.01 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1899gitlab # Enumerating objects: 5, done.1900gitlab # [ 418.366638] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:17:40 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 202 "" "git/2.55.0" 1191901gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1902gitlab # Delta compression using up to 4 threads1903gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1904gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 272 bytes | 68.00 KiB/s, done.1905gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1906gitlab # [ 419.127992] gitaly[966]: time="2026-09-13T10:17:41.036Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=301 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D4BJPHAMGG58F3SJZ3VXBD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-13T10:18:11.023" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-13T10:17:41.023" grpc.time_ms=13.037 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1907gitlab # [ 419.158212] gitaly[966]: time="2026-09-13T10:17:41.067Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M2D4BJPHAMGG58F3SJZ3VXBD duration_ms=220ns method=POST pid=966 status=200 url="http://unix/api/v4/internal/allowed"1908gitlab # [ 419.235739] gitaly[966]: time="2026-09-13T10:17:41.144Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M2D4BJPHAMGG58F3SJZ3VXBD duration_ms=76ns method=POST pid=966 status=200 url="http://unix/api/v4/internal/pre_receive"1909gitlab # [ 419.256679] gitaly[966]: time="2026-09-13T10:17:41.145Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D4BJPHAMGG58F3SJZ3VXBD grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5316 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:17:40.844" grpc.time_ms=301.028 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1910gitlab # [ 419.311567] gitaly[966]: time="2026-09-13T10:17:41.220Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D4BJPHAMGG58F3SJZ3VXBD grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4786 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:17:41.219" grpc.time_ms=1.135 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1911gitlab # [ 419.360092] gitaly[966]: time="2026-09-13T10:17:41.267Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D4BJPHAMGG58F3SJZ3VXBD grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4785 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:17:41.267" grpc.time_ms=0.62 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1912gitlab # [ 419.408093] gitaly[966]: time="2026-09-13T10:17:41.316Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D4BJPHAMGG58F3SJZ3VXBD grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:17:41.314" grpc.time_ms=1.844 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1913gitlab # [ 419.545664] gitaly[966]: time="2026-09-13T10:17:41.454Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M2D4BJPHAMGG58F3SJZ3VXBD duration_ms=87ns method=POST pid=966 status=200 url="http://unix/api/v4/internal/post_receive"1914gitlab # [ 419.556638] gitaly[966]: time="2026-09-13T10:17:41.455Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D4BJPHAMGG58F3SJZ3VXBD grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4810 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:17:41.365" grpc.time_ms=90.207 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1915gitlab # To http://gitlab/root/test-flake.git1916gitlab # [ 419.588273] gitaly[966]: time="2026-09-13T10:17:41.477Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M2D4BJPHAMGG58F3SJZ3VXBD diskcache=2c74eae4-502c-4204-99d9-3628610df334 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:17:40.673" pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1917gitlab # a97336d..456921b master -> master1918gitlab # [ 419.613466] gitlab-workhorse[885]: time="2026-09-13T10:17:41Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521919gitlab # [ 419.617183] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:17:41 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 11721920gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.64 seconds)1921gitlab: must succeed: git -C /tmp/test-flake rev-parse master1922gitlab # [ 419.622278] gitaly[966]: time="2026-09-13T10:17:41.477Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=399 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=847872 command.minflt=13346 command.oublock=32 command.real_time_ms=796 command.spawn_token_fork_ms=1 command.system_time_ms=243 command.user_time_ms=156 component=gitaly.StreamServerInterceptor correlation_id=01M2D4BJPHAMGG58F3SJZ3VXBD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=604 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:17:40.673" grpc.time_ms=803.787 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1923gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.02 seconds)1924gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=456921b37007e803dac208d4de47f6e13ba8b9e4'1925gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=456921b37007e803dac208d4de47f6e13ba8b9e4', in 0.14 seconds)1926gitlab # [ 419.785923] nixbot[3628]: INFO: ::ffff:127.0.0.1:51528 - "GET /api/repos/gitlab/root/test-flake/builds?commit=456921b37007e803dac208d4de47f6e13ba8b9e4 HTTP/1.1" 200 OK1927gitlab # [ 420.189880] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:42.097Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"06b21535c9f42df4fe7f12de","created_at":"2026-09-13T10:16:01.779Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a7a63caa2c92cfcb93af55dbe714d24c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:01.784Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-06b21535c9f42df4fe7f12de: start","job_status":"start","queue_duration_s":100.312783,"scheduling_latency_s":100.312841}1928gitlab # [ 420.377142] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:42.285Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"06b21535c9f42df4fe7f12de","created_at":"2026-09-13T10:16:01.779Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a7a63caa2c92cfcb93af55dbe714d24c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:01.784Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-06b21535c9f42df4fe7f12de: done: 0.188345 sec","job_status":"done","queue_duration_s":100.312783,"scheduling_latency_s":100.312841,"redis_calls":25,"redis_duration_s":0.008355,"redis_read_bytes":785,"redis_write_bytes":3128,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.00511,"redis_feature_flag_read_bytes":778,"redis_feature_flag_write_bytes":1966,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001826,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001419,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":11,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.037,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.032,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.072976,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00077202,"exclusive_lock_hold_duration_s":0.08982276,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.188345,"completed_at":"2026-09-13T10:17:42.285Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.067734,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1929gitlab # [ 420.485718] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:42.287Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"9dba8c953d8e19176f4571ef","created_at":"2026-09-13T10:16:01.791Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1f414dc057196439a88557ab1a6b46b5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:01.815Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-9dba8c953d8e19176f4571ef: start","job_status":"start","queue_duration_s":100.471872,"scheduling_latency_s":100.4719}1930gitlab # [ 420.505740] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:42.304Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1931gitlab # [ 420.509750] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:42.313Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"9dba8c953d8e19176f4571ef","created_at":"2026-09-13T10:16:01.791Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1f414dc057196439a88557ab1a6b46b5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:01.815Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-9dba8c953d8e19176f4571ef: done: 0.025891 sec","job_status":"done","queue_duration_s":100.471872,"scheduling_latency_s":100.4719,"redis_calls":11,"redis_duration_s":0.003223,"redis_read_bytes":9,"redis_write_bytes":1507,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001187,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":736,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001533,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000503,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014964,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025891,"completed_at":"2026-09-13T10:17:42.313Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001316,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1932gitlab # [ 420.553684] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:42.315Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"ec292ee9251049008cf2176c","created_at":"2026-09-13T10:16:01.836Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1510a3e99791e34f984c11eddf046159","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:01.839Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-ec292ee9251049008cf2176c: start","job_status":"start","queue_duration_s":100.47565,"scheduling_latency_s":100.475672}1933gitlab # [ 420.567994] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:42.340Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"ec292ee9251049008cf2176c","created_at":"2026-09-13T10:16:01.836Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1510a3e99791e34f984c11eddf046159","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:01.839Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-ec292ee9251049008cf2176c: done: 0.025712 sec","job_status":"done","queue_duration_s":100.47565,"scheduling_latency_s":100.475672,"redis_calls":13,"redis_duration_s":0.003304,"redis_read_bytes":11,"redis_write_bytes":1874,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001558,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1043,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001279,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000467,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015618,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025712,"completed_at":"2026-09-13T10:17:42.340Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001721,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1934gitlab # [ 420.609372] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:42.342Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"f6f417e5853eceb3fe354d22","created_at":"2026-09-13T10:16:01.845Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0dbdc96fe58c7c0699db6fcb002fe812","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:01.848Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-f6f417e5853eceb3fe354d22: start","job_status":"start","queue_duration_s":100.4943,"scheduling_latency_s":100.49432}1935gitlab # [ 420.622909] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:42.404Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"f6f417e5853eceb3fe354d22","created_at":"2026-09-13T10:16:01.845Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0dbdc96fe58c7c0699db6fcb002fe812","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:01.848Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-f6f417e5853eceb3fe354d22: done: 0.062315 sec","job_status":"done","queue_duration_s":100.4943,"scheduling_latency_s":100.49432,"redis_calls":17,"redis_duration_s":0.005031,"redis_read_bytes":15,"redis_write_bytes":2228,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.00326,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1517,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00129,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000481,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026761,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.062315,"completed_at":"2026-09-13T10:17:42.404Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.026875,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1936gitlab # [ 420.666379] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:42.407Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"256c04e96decf4c602c59c83","created_at":"2026-09-13T10:16:01.853Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"28f84cbfb138f9d5e3cc01fec4e81d22","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:01.855Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-256c04e96decf4c602c59c83: start","job_status":"start","queue_duration_s":100.551636,"scheduling_latency_s":100.551661}1937gitlab # [ 420.680929] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:42.444Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"256c04e96decf4c602c59c83","created_at":"2026-09-13T10:16:01.853Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"28f84cbfb138f9d5e3cc01fec4e81d22","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:01.855Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-256c04e96decf4c602c59c83: done: 0.037527 sec","job_status":"done","queue_duration_s":100.551636,"scheduling_latency_s":100.551661,"redis_calls":13,"redis_duration_s":0.003676,"redis_read_bytes":11,"redis_write_bytes":1780,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001127,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":708,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00132,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001229,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020719,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00061518,"exclusive_lock_hold_duration_s":0.01127532,"duration_s":0.037527,"completed_at":"2026-09-13T10:17:42.444Z","load_balancing_strategy":"primary","db_duration_s":0.007935,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1938gitlab # [ 420.721860] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:42.446Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"908b65540f4ed404d678c826","created_at":"2026-09-13T10:16:01.874Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ffaf1148905aba945a42c9761b1a38a7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:01.880Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-908b65540f4ed404d678c826: start","job_status":"start","queue_duration_s":100.565963,"scheduling_latency_s":100.565983}1939gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=456921b37007e803dac208d4de47f6e13ba8b9e4'1940gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=456921b37007e803dac208d4de47f6e13ba8b9e4', in 0.06 seconds)1941gitlab # [ 420.847888] nixbot[3628]: INFO: ::ffff:127.0.0.1:35058 - "GET /api/repos/gitlab/root/test-flake/builds?commit=456921b37007e803dac208d4de47f6e13ba8b9e4 HTTP/1.1" 200 OK1942gitlab # [ 420.938742] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:42.847Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"908b65540f4ed404d678c826","created_at":"2026-09-13T10:16:01.874Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ffaf1148905aba945a42c9761b1a38a7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:01.880Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-908b65540f4ed404d678c826: done: 0.400588 sec","job_status":"done","queue_duration_s":100.565963,"scheduling_latency_s":100.565983,"redis_calls":13,"redis_duration_s":0.004103,"redis_read_bytes":11,"redis_write_bytes":1651,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001102,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":664,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001616,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001385,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.059,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.057,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.328017,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00058748,"exclusive_lock_hold_duration_s":0.3711823,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.400588,"completed_at":"2026-09-13T10:17:42.846Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.089259,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1943gitlab # [ 421.033587] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:42.849Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleDeleteObjectsCronWorker","jid":"989bf764a64fb45c2541e3cd","created_at":"2026-09-13T10:16:01.898Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8251d4193fad694e50c4958d17fdca56b6fb205fa6961a4b1ccb7cd497bc09c1","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:01.913Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleDeleteObjectsCronWorker JID-989bf764a64fb45c2541e3cd: start","job_status":"start","queue_duration_s":100.936125,"scheduling_latency_s":100.936153}1944gitlab # [ 421.070116] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:42.978Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleDeleteObjectsCronWorker","jid":"989bf764a64fb45c2541e3cd","created_at":"2026-09-13T10:16:01.898Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8251d4193fad694e50c4958d17fdca56b6fb205fa6961a4b1ccb7cd497bc09c1","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:01.913Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleDeleteObjectsCronWorker JID-989bf764a64fb45c2541e3cd: done: 0.129358 sec","job_status":"done","queue_duration_s":100.936125,"scheduling_latency_s":100.936153,"redis_calls":67,"redis_duration_s":0.020118999999999998,"redis_read_bytes":117,"redis_write_bytes":41699,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001811,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1023,"redis_queues_calls":2,"redis_queues_duration_s":0.000981,"redis_queues_read_bytes":4,"redis_queues_write_bytes":37035,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.01656,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3374,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000767,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":267,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.10657,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00037724,"duration_s":0.129358,"completed_at":"2026-09-13T10:17:42.978Z","load_balancing_strategy":"primary","db_duration_s":0.001838,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1945gitlab # [ 421.140753] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:42.980Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"d68c55407d43af7b1cdce3fa","created_at":"2026-09-13T10:16:01.934Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2f11d83ed6d729523d1a9e69390a7e95","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:01.944Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-d68c55407d43af7b1cdce3fa: start","job_status":"start","queue_duration_s":101.035663,"scheduling_latency_s":101.035681}1946gitlab # [ 421.156265] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.002Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"d68c55407d43af7b1cdce3fa","created_at":"2026-09-13T10:16:01.934Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2f11d83ed6d729523d1a9e69390a7e95","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:01.944Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-d68c55407d43af7b1cdce3fa: done: 0.022264 sec","job_status":"done","queue_duration_s":101.035663,"scheduling_latency_s":101.035681,"redis_calls":13,"redis_duration_s":0.003168,"redis_read_bytes":10,"redis_write_bytes":1453,"redis_cache_calls":2,"redis_cache_duration_s":0.000508,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001014,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":664,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001196,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":482,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00045,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":214,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013239,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022264,"completed_at":"2026-09-13T10:17:43.002Z","load_balancing_strategy":"primary","db_duration_s":0.001356,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1947gitlab # [ 421.198908] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.004Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"bc097bbf190ef4dc96476701","created_at":"2026-09-13T10:16:01.965Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7045e320d68985de0dc0078ce6f3f7f2","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:01.975Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-bc097bbf190ef4dc96476701: start","job_status":"start","queue_duration_s":101.028891,"scheduling_latency_s":101.028909}1948gitlab # [ 421.212899] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.059Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"bc097bbf190ef4dc96476701","created_at":"2026-09-13T10:16:01.965Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7045e320d68985de0dc0078ce6f3f7f2","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:01.975Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-bc097bbf190ef4dc96476701: done: 0.054904 sec","job_status":"done","queue_duration_s":101.028891,"scheduling_latency_s":101.028909,"redis_calls":11,"redis_duration_s":0.003812,"redis_read_bytes":9,"redis_write_bytes":1341,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001265,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":660,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001985,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000562,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037568,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.054904,"completed_at":"2026-09-13T10:17:43.058Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00587,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1949gitlab # [ 421.254640] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.062Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"e8764f5ed948baf6ad1fd407","created_at":"2026-09-13T10:16:02.001Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9fe6ac5f89ba432fe0be81b9c73c468c","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.006Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-e8764f5ed948baf6ad1fd407: start","job_status":"start","queue_duration_s":101.056327,"scheduling_latency_s":101.056376}1950gitlab # [ 421.268363] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.140Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"e8764f5ed948baf6ad1fd407","created_at":"2026-09-13T10:16:02.001Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9fe6ac5f89ba432fe0be81b9c73c468c","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.006Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-e8764f5ed948baf6ad1fd407: done: 0.077812 sec","job_status":"done","queue_duration_s":101.056327,"scheduling_latency_s":101.056376,"redis_calls":11,"redis_duration_s":0.004368,"redis_read_bytes":9,"redis_write_bytes":1594,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001308,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":768,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00219,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00087,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040976,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.077812,"completed_at":"2026-09-13T10:17:43.140Z","load_balancing_strategy":"primary","db_duration_s":0.020893,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1951gitlab # [ 421.309971] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.143Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"8b2aa9ce6647b9b2a25dfa52","created_at":"2026-09-13T10:16:02.010Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"125543b283ca4ab5ac55094681cc15bd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.014Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-8b2aa9ce6647b9b2a25dfa52: start","job_status":"start","queue_duration_s":101.129106,"scheduling_latency_s":101.129147}1952gitlab # [ 421.324287] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.232Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"8b2aa9ce6647b9b2a25dfa52","created_at":"2026-09-13T10:16:02.010Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"125543b283ca4ab5ac55094681cc15bd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.014Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-8b2aa9ce6647b9b2a25dfa52: done: 0.088587 sec","job_status":"done","queue_duration_s":101.129106,"scheduling_latency_s":101.129147,"redis_calls":13,"redis_duration_s":0.004846,"redis_read_bytes":11,"redis_write_bytes":1921,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001241,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":752,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001986,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001619,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.027,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.045555,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00101466,"exclusive_lock_hold_duration_s":0.03020044,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.088587,"completed_at":"2026-09-13T10:17:43.231Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.028465,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1953gitlab # [ 421.368001] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.235Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"a2bade485e07bfcd31c50b00","created_at":"2026-09-13T10:16:02.017Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d6565c7133e48a4ede472056712e9e9a","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.023Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-a2bade485e07bfcd31c50b00: start","job_status":"start","queue_duration_s":101.211961,"scheduling_latency_s":101.211996}1954gitlab # [ 421.382032] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.289Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"a2bade485e07bfcd31c50b00","created_at":"2026-09-13T10:16:02.017Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d6565c7133e48a4ede472056712e9e9a","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.023Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-a2bade485e07bfcd31c50b00: done: 0.054199 sec","job_status":"done","queue_duration_s":101.211961,"scheduling_latency_s":101.211996,"redis_calls":13,"redis_duration_s":0.005189,"redis_read_bytes":11,"redis_write_bytes":1803,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001216,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":716,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00207,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001903,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034273,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00129272,"exclusive_lock_hold_duration_s":0.00687176,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.054199,"completed_at":"2026-09-13T10:17:43.289Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003755,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1955gitlab # [ 421.423998] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.292Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"ed8620b1f9147f5fa7bf07f9","created_at":"2026-09-13T10:16:02.031Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"bee8d51923ccb0a9b559c755e679c742","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.040Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-ed8620b1f9147f5fa7bf07f9: start","job_status":"start","queue_duration_s":101.252441,"scheduling_latency_s":101.252484}1956gitlab # [ 421.437584] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.344Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"ed8620b1f9147f5fa7bf07f9","created_at":"2026-09-13T10:16:02.031Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"bee8d51923ccb0a9b559c755e679c742","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.040Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-ed8620b1f9147f5fa7bf07f9: done: 0.051782 sec","job_status":"done","queue_duration_s":101.252441,"scheduling_latency_s":101.252484,"redis_calls":12,"redis_duration_s":0.005647,"redis_read_bytes":11,"redis_write_bytes":1594,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.0024,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":728,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002333,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000914,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032903,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00067288,"duration_s":0.051782,"completed_at":"2026-09-13T10:17:43.344Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001863,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1957gitlab # [ 421.479139] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.347Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"e03ee5eef9499bfdbede80a3","created_at":"2026-09-13T10:16:02.043Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4854a360e5b2886c87ae10fa5441f090","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.047Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-e03ee5eef9499bfdbede80a3: start","job_status":"start","queue_duration_s":101.299382,"scheduling_latency_s":101.299412}1958gitlab # [ 421.496025] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.404Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"e03ee5eef9499bfdbede80a3","created_at":"2026-09-13T10:16:02.043Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4854a360e5b2886c87ae10fa5441f090","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.047Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-e03ee5eef9499bfdbede80a3: done: 0.057377 sec","job_status":"done","queue_duration_s":101.299382,"scheduling_latency_s":101.299412,"redis_calls":14,"redis_duration_s":0.004802,"redis_read_bytes":12,"redis_write_bytes":1967,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001315,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":728,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002093,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001394,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03923,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.015703,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.057377,"completed_at":"2026-09-13T10:17:43.404Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005777,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1959gitlab # [ 421.539293] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.406Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"b9d82f2256f092840de9aa19","created_at":"2026-09-13T10:16:02.061Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"384d33983851eb3e087a24a202180143","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.064Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-b9d82f2256f092840de9aa19: start","job_status":"start","queue_duration_s":101.341672,"scheduling_latency_s":101.341698}1960gitlab # [ 421.554142] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.451Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"b9d82f2256f092840de9aa19","created_at":"2026-09-13T10:16:02.061Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"384d33983851eb3e087a24a202180143","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.064Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-b9d82f2256f092840de9aa19: done: 0.04483 sec","job_status":"done","queue_duration_s":101.341672,"scheduling_latency_s":101.341698,"redis_calls":11,"redis_duration_s":0.00379,"redis_read_bytes":9,"redis_write_bytes":1522,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001509,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":736,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001664,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000617,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027593,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.04483,"completed_at":"2026-09-13T10:17:43.451Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006628,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1961gitlab # [ 421.595491] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.453Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"1a2d612601974c09f066b84c","created_at":"2026-09-13T10:16:02.067Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"354b81e573038e37386415cd2bc2d438","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.071Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-1a2d612601974c09f066b84c: start","job_status":"start","queue_duration_s":101.38224,"scheduling_latency_s":101.382267}1962gitlab # [ 421.609973] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.472Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1963gitlab # [ 421.613532] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.481Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"1a2d612601974c09f066b84c","created_at":"2026-09-13T10:16:02.067Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"354b81e573038e37386415cd2bc2d438","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.071Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-1a2d612601974c09f066b84c: done: 0.027902 sec","job_status":"done","queue_duration_s":101.38224,"scheduling_latency_s":101.382267,"redis_calls":11,"redis_duration_s":0.003039,"redis_read_bytes":9,"redis_write_bytes":1498,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001115,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":732,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001384,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00054,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016922,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027902,"completed_at":"2026-09-13T10:17:43.481Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001249,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1964gitlab # [ 421.654965] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.483Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"48d3f2f57c0eabbbb49933e7","created_at":"2026-09-13T10:16:02.076Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ec4f485f45cf90c3d8886d99dbf1751e","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.079Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-48d3f2f57c0eabbbb49933e7: start","job_status":"start","queue_duration_s":101.403954,"scheduling_latency_s":101.403976}1965gitlab # [ 421.670512] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.513Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"48d3f2f57c0eabbbb49933e7","created_at":"2026-09-13T10:16:02.076Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-dbd659909a8d4a85","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ec4f485f45cf90c3d8886d99dbf1751e","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.079Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-48d3f2f57c0eabbbb49933e7: done: 0.030097 sec","job_status":"done","queue_duration_s":101.403954,"scheduling_latency_s":101.403976,"redis_calls":13,"redis_duration_s":0.003491,"redis_read_bytes":11,"redis_write_bytes":1855,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001011,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":728,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001316,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001164,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":591,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017341,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00057228,"exclusive_lock_hold_duration_s":0.00599518,"duration_s":0.030097,"completed_at":"2026-09-13T10:17:43.513Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004475,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1966gitlab # [ 421.712134] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.515Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"211ba6a96abc1ce3b4becc7f","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-211ba6a96abc1ce3b4becc7f: start","job_status":"start","queue_duration_s":101.187201,"scheduling_latency_s":101.18722}1967gitlab # [ 421.724341] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.549Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"211ba6a96abc1ce3b4becc7f","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-211ba6a96abc1ce3b4becc7f: done: 0.033842 sec","job_status":"done","queue_duration_s":101.187201,"scheduling_latency_s":101.18722,"redis_calls":23,"redis_duration_s":0.005818,"redis_read_bytes":20,"redis_write_bytes":2604,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002505,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1447,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001155,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002158,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021194,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.033842,"completed_at":"2026-09-13T10:17:43.549Z","load_balancing_strategy":"primary","db_duration_s":0.003192,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1968gitlab # [ 421.764448] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.550Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"acf3b9513183653f198c6ec7","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-acf3b9513183653f198c6ec7: start","job_status":"start","queue_duration_s":101.222497,"scheduling_latency_s":101.222516}1969gitlab # [ 421.776573] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.565Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"acf3b9513183653f198c6ec7","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-acf3b9513183653f198c6ec7: done: 0.01517 sec","job_status":"done","queue_duration_s":101.222497,"scheduling_latency_s":101.222516,"redis_calls":13,"redis_duration_s":0.0028179999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000935,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001883,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009398,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01517,"completed_at":"2026-09-13T10:17:43.565Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1970gitlab # [ 421.815415] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.567Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ddfa2897f658668349d9f0a","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ddfa2897f658668349d9f0a: start","job_status":"start","queue_duration_s":101.239134,"scheduling_latency_s":101.239151}1971gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=456921b37007e803dac208d4de47f6e13ba8b9e4'1972gitlab # [ 421.827804] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.582Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ddfa2897f658668349d9f0a","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ddfa2897f658668349d9f0a: done: 0.015451 sec","job_status":"done","queue_duration_s":101.239134,"scheduling_latency_s":101.239151,"redis_calls":13,"redis_duration_s":0.003007,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000913,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002094,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009618,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015451,"completed_at":"2026-09-13T10:17:43.582Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1973gitlab # [ 421.866144] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.584Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cb8c74b62e8b4113502c870e","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cb8c74b62e8b4113502c870e: start","job_status":"start","queue_duration_s":101.255964,"scheduling_latency_s":101.25598}1974gitlab # [ 421.877458] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.599Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cb8c74b62e8b4113502c870e","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cb8c74b62e8b4113502c870e: done: 0.01512 sec","job_status":"done","queue_duration_s":101.255964,"scheduling_latency_s":101.25598,"redis_calls":13,"redis_duration_s":0.002906,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000944,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001962,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009301,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01512,"completed_at":"2026-09-13T10:17:43.599Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1975gitlab # [ 421.915211] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.600Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f48c268a64035e8b5e24c95","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f48c268a64035e8b5e24c95: start","job_status":"start","queue_duration_s":101.272542,"scheduling_latency_s":101.272558}1976gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=456921b37007e803dac208d4de47f6e13ba8b9e4', in 0.09 seconds)1977gitlab # [ 421.927638] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.615Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f48c268a64035e8b5e24c95","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f48c268a64035e8b5e24c95: done: 0.014977 sec","job_status":"done","queue_duration_s":101.272542,"scheduling_latency_s":101.272558,"redis_calls":13,"redis_duration_s":0.0028280000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000893,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001935,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009136,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014977,"completed_at":"2026-09-13T10:17:43.615Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1978gitlab # [ 421.966389] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.617Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7fd0e3a650492096dd471e13","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7fd0e3a650492096dd471e13: start","job_status":"start","queue_duration_s":101.288775,"scheduling_latency_s":101.28879}1979gitlab # [ 421.978478] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.632Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7fd0e3a650492096dd471e13","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7fd0e3a650492096dd471e13: done: 0.014966 sec","job_status":"done","queue_duration_s":101.288775,"scheduling_latency_s":101.28879,"redis_calls":13,"redis_duration_s":0.0028770000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000979,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001898,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009326,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014966,"completed_at":"2026-09-13T10:17:43.631Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1980gitlab # [ 422.017187] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.633Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"61eee7f09e4ffcd37aa36ffe","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-61eee7f09e4ffcd37aa36ffe: start","job_status":"start","queue_duration_s":101.305053,"scheduling_latency_s":101.305069}1981gitlab # [ 422.029788] nixbot[3628]: INFO: ::ffff:127.0.0.1:35062 - "GET /api/repos/gitlab/root/test-flake/builds?commit=456921b37007e803dac208d4de47f6e13ba8b9e4 HTTP/1.1" 200 OK1982gitlab # [ 422.031685] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.647Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"61eee7f09e4ffcd37aa36ffe","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-61eee7f09e4ffcd37aa36ffe: done: 0.014605 sec","job_status":"done","queue_duration_s":101.305053,"scheduling_latency_s":101.305069,"redis_calls":13,"redis_duration_s":0.002529,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000904,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001625,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009193,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014605,"completed_at":"2026-09-13T10:17:43.647Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1983gitlab # [ 422.071035] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.649Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3905e4786c77be3b295185c8","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3905e4786c77be3b295185c8: start","job_status":"start","queue_duration_s":101.320983,"scheduling_latency_s":101.320999}1984gitlab # [ 422.083884] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.664Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3905e4786c77be3b295185c8","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3905e4786c77be3b295185c8: done: 0.014784 sec","job_status":"done","queue_duration_s":101.320983,"scheduling_latency_s":101.320999,"redis_calls":13,"redis_duration_s":0.002477,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000896,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001581,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009288,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014784,"completed_at":"2026-09-13T10:17:43.663Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1985gitlab # [ 422.122938] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.665Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ade7573a106dd1df3c63fde","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ade7573a106dd1df3c63fde: start","job_status":"start","queue_duration_s":101.337137,"scheduling_latency_s":101.337153}1986gitlab # [ 422.135248] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.680Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ade7573a106dd1df3c63fde","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ade7573a106dd1df3c63fde: done: 0.014724 sec","job_status":"done","queue_duration_s":101.337137,"scheduling_latency_s":101.337153,"redis_calls":13,"redis_duration_s":0.002712,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000984,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001728,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009302,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014724,"completed_at":"2026-09-13T10:17:43.680Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1987gitlab # [ 422.174237] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.681Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a4ddeb106ebeb1ff1a53f79b","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a4ddeb106ebeb1ff1a53f79b: start","job_status":"start","queue_duration_s":101.353294,"scheduling_latency_s":101.35331}1988gitlab # [ 422.186543] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.696Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a4ddeb106ebeb1ff1a53f79b","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a4ddeb106ebeb1ff1a53f79b: done: 0.01462 sec","job_status":"done","queue_duration_s":101.353294,"scheduling_latency_s":101.35331,"redis_calls":13,"redis_duration_s":0.002635,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000901,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001734,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009295,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01462,"completed_at":"2026-09-13T10:17:43.696Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1989gitlab # [ 422.225349] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.697Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6828bdff440c71fac3c2b95","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6828bdff440c71fac3c2b95: start","job_status":"start","queue_duration_s":101.369408,"scheduling_latency_s":101.369425}1990gitlab # [ 422.237731] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.712Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6828bdff440c71fac3c2b95","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6828bdff440c71fac3c2b95: done: 0.015005 sec","job_status":"done","queue_duration_s":101.369408,"scheduling_latency_s":101.369425,"redis_calls":13,"redis_duration_s":0.002752,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000939,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001813,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009416,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015005,"completed_at":"2026-09-13T10:17:43.712Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1991gitlab # [ 422.277413] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.714Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"53120571fe03f9e9a7cd7543","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-53120571fe03f9e9a7cd7543: start","job_status":"start","queue_duration_s":101.38574,"scheduling_latency_s":101.385757}1992gitlab # [ 422.289464] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.728Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"53120571fe03f9e9a7cd7543","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-53120571fe03f9e9a7cd7543: done: 0.014821 sec","job_status":"done","queue_duration_s":101.38574,"scheduling_latency_s":101.385757,"redis_calls":13,"redis_duration_s":0.002663,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000901,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001762,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009289,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014821,"completed_at":"2026-09-13T10:17:43.728Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1993gitlab # [ 422.328211] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.730Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b7be9f26e6a8d939b13b9fe","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b7be9f26e6a8d939b13b9fe: start","job_status":"start","queue_duration_s":101.401938,"scheduling_latency_s":101.401955}1994gitlab # [ 422.340417] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.745Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b7be9f26e6a8d939b13b9fe","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b7be9f26e6a8d939b13b9fe: done: 0.014961 sec","job_status":"done","queue_duration_s":101.401938,"scheduling_latency_s":101.401955,"redis_calls":13,"redis_duration_s":0.002757,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000914,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001843,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009317,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014961,"completed_at":"2026-09-13T10:17:43.745Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1995gitlab # [ 422.378867] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.746Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a52b20b4bf3b091dbca0b4d1","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a52b20b4bf3b091dbca0b4d1: start","job_status":"start","queue_duration_s":101.418331,"scheduling_latency_s":101.418346}1996gitlab # [ 422.391035] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.761Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a52b20b4bf3b091dbca0b4d1","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a52b20b4bf3b091dbca0b4d1: done: 0.015045 sec","job_status":"done","queue_duration_s":101.418331,"scheduling_latency_s":101.418346,"redis_calls":13,"redis_duration_s":0.002784,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000992,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001792,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00924,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015045,"completed_at":"2026-09-13T10:17:43.761Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1997gitlab # [ 422.429868] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.763Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"87ea083e290ae1290b2bd61e","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-87ea083e290ae1290b2bd61e: start","job_status":"start","queue_duration_s":101.434758,"scheduling_latency_s":101.434774}1998gitlab # [ 422.442148] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.778Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"87ea083e290ae1290b2bd61e","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-87ea083e290ae1290b2bd61e: done: 0.015112 sec","job_status":"done","queue_duration_s":101.434758,"scheduling_latency_s":101.434774,"redis_calls":13,"redis_duration_s":0.0026190000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000934,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001685,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009507,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015112,"completed_at":"2026-09-13T10:17:43.778Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1999gitlab # [ 422.481310] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.779Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0f21c87ee98c422fea560756","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0f21c87ee98c422fea560756: start","job_status":"start","queue_duration_s":101.45132,"scheduling_latency_s":101.451335}2000gitlab # [ 422.493603] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.794Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0f21c87ee98c422fea560756","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0f21c87ee98c422fea560756: done: 0.014958 sec","job_status":"done","queue_duration_s":101.45132,"scheduling_latency_s":101.451335,"redis_calls":13,"redis_duration_s":0.002798,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000923,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001875,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009235,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014958,"completed_at":"2026-09-13T10:17:43.794Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2001gitlab # [ 422.532049] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.796Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7580fe5ef07d798b033b69de","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7580fe5ef07d798b033b69de: start","job_status":"start","queue_duration_s":101.467737,"scheduling_latency_s":101.467755}2002gitlab # [ 422.544122] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.815Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7580fe5ef07d798b033b69de","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7580fe5ef07d798b033b69de: done: 0.019149 sec","job_status":"done","queue_duration_s":101.467737,"scheduling_latency_s":101.467755,"redis_calls":13,"redis_duration_s":0.002627,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001032,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001595,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009799,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019149,"completed_at":"2026-09-13T10:17:43.815Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2003gitlab # [ 422.582313] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.816Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"265f3820838939f910d1df0e","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-265f3820838939f910d1df0e: start","job_status":"start","queue_duration_s":101.488288,"scheduling_latency_s":101.488305}2004gitlab # [ 422.594365] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.832Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"265f3820838939f910d1df0e","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-265f3820838939f910d1df0e: done: 0.015724 sec","job_status":"done","queue_duration_s":101.488288,"scheduling_latency_s":101.488305,"redis_calls":13,"redis_duration_s":0.00251,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000973,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001537,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010164,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015724,"completed_at":"2026-09-13T10:17:43.832Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2005gitlab # [ 422.632608] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.833Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"11b8197580edbe8ef671ddc6","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-11b8197580edbe8ef671ddc6: start","job_status":"start","queue_duration_s":101.505519,"scheduling_latency_s":101.505536}2006gitlab # [ 422.644766] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.849Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"11b8197580edbe8ef671ddc6","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-11b8197580edbe8ef671ddc6: done: 0.015958 sec","job_status":"done","queue_duration_s":101.505519,"scheduling_latency_s":101.505536,"redis_calls":13,"redis_duration_s":0.00334,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001257,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002083,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009869,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015958,"completed_at":"2026-09-13T10:17:43.849Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2007gitlab # [ 422.684602] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.851Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bdb87210c305be6023920584","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bdb87210c305be6023920584: start","job_status":"start","queue_duration_s":101.52311,"scheduling_latency_s":101.523126}2008gitlab # [ 422.696704] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.866Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bdb87210c305be6023920584","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bdb87210c305be6023920584: done: 0.014806 sec","job_status":"done","queue_duration_s":101.52311,"scheduling_latency_s":101.523126,"redis_calls":13,"redis_duration_s":0.0026349999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000922,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001713,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009361,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014806,"completed_at":"2026-09-13T10:17:43.866Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2009gitlab # [ 422.734894] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.867Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"592955fc3d5d8b6dc21ea0d7","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-592955fc3d5d8b6dc21ea0d7: start","job_status":"start","queue_duration_s":101.539349,"scheduling_latency_s":101.539368}2010gitlab # [ 422.747209] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.882Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"592955fc3d5d8b6dc21ea0d7","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-592955fc3d5d8b6dc21ea0d7: done: 0.014836 sec","job_status":"done","queue_duration_s":101.539349,"scheduling_latency_s":101.539368,"redis_calls":13,"redis_duration_s":0.002613,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000898,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001715,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009296,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014836,"completed_at":"2026-09-13T10:17:43.882Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2011gitlab # [ 422.785501] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.883Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12ce09eb468df324e10c4843","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12ce09eb468df324e10c4843: start","job_status":"start","queue_duration_s":101.555517,"scheduling_latency_s":101.555533}2012gitlab # [ 422.798005] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.898Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12ce09eb468df324e10c4843","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12ce09eb468df324e10c4843: done: 0.014829 sec","job_status":"done","queue_duration_s":101.555517,"scheduling_latency_s":101.555533,"redis_calls":13,"redis_duration_s":0.002594,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000946,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001648,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009384,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014829,"completed_at":"2026-09-13T10:17:43.898Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2013gitlab # [ 422.836127] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.900Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8f2895352cffad2f8ec2db1d","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8f2895352cffad2f8ec2db1d: start","job_status":"start","queue_duration_s":101.571727,"scheduling_latency_s":101.571743}2014gitlab # [ 422.847998] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.914Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8f2895352cffad2f8ec2db1d","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8f2895352cffad2f8ec2db1d: done: 0.014605 sec","job_status":"done","queue_duration_s":101.571727,"scheduling_latency_s":101.571743,"redis_calls":13,"redis_duration_s":0.002464,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000912,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001552,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009299,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014605,"completed_at":"2026-09-13T10:17:43.914Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2015gitlab # [ 422.886956] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.916Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b3df725f64f51657c9ec735b","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b3df725f64f51657c9ec735b: start","job_status":"start","queue_duration_s":101.58771,"scheduling_latency_s":101.587727}2016gitlab # [ 422.899236] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.930Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b3df725f64f51657c9ec735b","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b3df725f64f51657c9ec735b: done: 0.014948 sec","job_status":"done","queue_duration_s":101.58771,"scheduling_latency_s":101.587727,"redis_calls":13,"redis_duration_s":0.002698,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000922,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001776,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009174,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014948,"completed_at":"2026-09-13T10:17:43.930Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2017gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=456921b37007e803dac208d4de47f6e13ba8b9e4'2018gitlab # [ 422.937349] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.932Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0f6ae1a2bd4b02f884581d5a","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0f6ae1a2bd4b02f884581d5a: start","job_status":"start","queue_duration_s":101.604014,"scheduling_latency_s":101.604029}2019gitlab # [ 422.948867] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.947Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0f6ae1a2bd4b02f884581d5a","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0f6ae1a2bd4b02f884581d5a: done: 0.014954 sec","job_status":"done","queue_duration_s":101.604014,"scheduling_latency_s":101.604029,"redis_calls":13,"redis_duration_s":0.002783,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000904,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001879,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009176,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014954,"completed_at":"2026-09-13T10:17:43.947Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2020gitlab # [ 422.984520] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.948Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5af6a4bc06bef93c59fb0b9c","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5af6a4bc06bef93c59fb0b9c: start","job_status":"start","queue_duration_s":101.620341,"scheduling_latency_s":101.620358}2021gitlab # [ 422.996677] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.963Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5af6a4bc06bef93c59fb0b9c","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5af6a4bc06bef93c59fb0b9c: done: 0.014871 sec","job_status":"done","queue_duration_s":101.620341,"scheduling_latency_s":101.620358,"redis_calls":13,"redis_duration_s":0.002775,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00088,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001895,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009207,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014871,"completed_at":"2026-09-13T10:17:43.963Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2022gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=456921b37007e803dac208d4de47f6e13ba8b9e4', in 0.10 seconds)2023gitlab # [ 423.034855] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.964Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"36eae6a94f67ac210c918a5f","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-36eae6a94f67ac210c918a5f: start","job_status":"start","queue_duration_s":101.636618,"scheduling_latency_s":101.636633}2024gitlab # [ 423.047089] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.979Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"36eae6a94f67ac210c918a5f","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-36eae6a94f67ac210c918a5f: done: 0.014737 sec","job_status":"done","queue_duration_s":101.636618,"scheduling_latency_s":101.636633,"redis_calls":13,"redis_duration_s":0.0026190000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000947,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001672,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009263,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014737,"completed_at":"2026-09-13T10:17:43.979Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2025gitlab # [ 423.085343] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.981Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4f03ef7efe7ad6a56c600456","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4f03ef7efe7ad6a56c600456: start","job_status":"start","queue_duration_s":101.65281,"scheduling_latency_s":101.652825}2026gitlab # [ 423.098933] nixbot[3628]: INFO: ::ffff:127.0.0.1:35068 - "GET /api/repos/gitlab/root/test-flake/builds?commit=456921b37007e803dac208d4de47f6e13ba8b9e4 HTTP/1.1" 200 OK2027gitlab # [ 423.100909] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.995Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4f03ef7efe7ad6a56c600456","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4f03ef7efe7ad6a56c600456: done: 0.014644 sec","job_status":"done","queue_duration_s":101.65281,"scheduling_latency_s":101.652825,"redis_calls":13,"redis_duration_s":0.002537,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000927,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00161,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009269,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014644,"completed_at":"2026-09-13T10:17:43.995Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2028gitlab # [ 423.139226] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:43.997Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7913a6f6bd2bf4058effc53a","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7913a6f6bd2bf4058effc53a: start","job_status":"start","queue_duration_s":101.668766,"scheduling_latency_s":101.668782}2029gitlab # [ 423.151345] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.011Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7913a6f6bd2bf4058effc53a","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7913a6f6bd2bf4058effc53a: done: 0.014599 sec","job_status":"done","queue_duration_s":101.668766,"scheduling_latency_s":101.668782,"redis_calls":13,"redis_duration_s":0.002567,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000887,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00168,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009244,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014599,"completed_at":"2026-09-13T10:17:44.011Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2030gitlab # [ 423.189629] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.013Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"26acce8ff7b6cc5408b3eb6f","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-26acce8ff7b6cc5408b3eb6f: start","job_status":"start","queue_duration_s":101.684712,"scheduling_latency_s":101.684728}2031gitlab # [ 423.201741] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.027Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"26acce8ff7b6cc5408b3eb6f","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-26acce8ff7b6cc5408b3eb6f: done: 0.014779 sec","job_status":"done","queue_duration_s":101.684712,"scheduling_latency_s":101.684728,"redis_calls":13,"redis_duration_s":0.002735,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000993,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001742,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009366,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014779,"completed_at":"2026-09-13T10:17:44.027Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2032gitlab # [ 423.240437] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.029Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"43d0124c6a1d8945830a0332","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-43d0124c6a1d8945830a0332: start","job_status":"start","queue_duration_s":101.700911,"scheduling_latency_s":101.700945}2033gitlab # [ 423.252530] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.044Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"43d0124c6a1d8945830a0332","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-43d0124c6a1d8945830a0332: done: 0.015087 sec","job_status":"done","queue_duration_s":101.700911,"scheduling_latency_s":101.700945,"redis_calls":13,"redis_duration_s":0.002767,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000951,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001816,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009344,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015087,"completed_at":"2026-09-13T10:17:44.044Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2034gitlab # [ 423.290601] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.045Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8b9ab9fc8bea7532e4490d83","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8b9ab9fc8bea7532e4490d83: start","job_status":"start","queue_duration_s":101.717415,"scheduling_latency_s":101.717434}2035gitlab # [ 423.303989] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.060Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8b9ab9fc8bea7532e4490d83","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8b9ab9fc8bea7532e4490d83: done: 0.014996 sec","job_status":"done","queue_duration_s":101.717415,"scheduling_latency_s":101.717434,"redis_calls":13,"redis_duration_s":0.00274,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000929,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001811,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009411,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014996,"completed_at":"2026-09-13T10:17:44.060Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2036gitlab # [ 423.342003] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.061Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"11133e639e1f1311fb7d7247","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-11133e639e1f1311fb7d7247: start","job_status":"start","queue_duration_s":101.733657,"scheduling_latency_s":101.733674}2037gitlab # [ 423.354016] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.076Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"11133e639e1f1311fb7d7247","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-11133e639e1f1311fb7d7247: done: 0.01476 sec","job_status":"done","queue_duration_s":101.733657,"scheduling_latency_s":101.733674,"redis_calls":13,"redis_duration_s":0.002708,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00089,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001818,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009144,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01476,"completed_at":"2026-09-13T10:17:44.076Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2038gitlab # [ 423.392426] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.077Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8a36c1b19f999f0ce5d56c4d","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8a36c1b19f999f0ce5d56c4d: start","job_status":"start","queue_duration_s":101.749695,"scheduling_latency_s":101.749712}2039gitlab # [ 423.404358] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.092Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8a36c1b19f999f0ce5d56c4d","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8a36c1b19f999f0ce5d56c4d: done: 0.014984 sec","job_status":"done","queue_duration_s":101.749695,"scheduling_latency_s":101.749712,"redis_calls":13,"redis_duration_s":0.002666,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000961,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001705,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009246,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014984,"completed_at":"2026-09-13T10:17:44.092Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2040gitlab # [ 423.442466] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.094Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"31ab6c6c66a59449e8d755c4","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-31ab6c6c66a59449e8d755c4: start","job_status":"start","queue_duration_s":101.76608,"scheduling_latency_s":101.766096}2041gitlab # [ 423.455076] gitaly[966]: time="2026-09-13T10:17:45.275Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=576 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2D4929ZQRKFKW6TJVV0RNR0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:17:55.001" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:17:45.262" grpc.time_ms=13.426 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2042gitlab # [ 423.467984] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.109Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"31ab6c6c66a59449e8d755c4","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-31ab6c6c66a59449e8d755c4: done: 0.014881 sec","job_status":"done","queue_duration_s":101.76608,"scheduling_latency_s":101.766096,"redis_calls":13,"redis_duration_s":0.002807,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000909,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001898,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009185,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014881,"completed_at":"2026-09-13T10:17:44.109Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2043gitlab # [ 423.507485] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.110Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c82363e5a93a803cfcfa1edc","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c82363e5a93a803cfcfa1edc: start","job_status":"start","queue_duration_s":101.782292,"scheduling_latency_s":101.782308}2044gitlab # [ 423.519790] gitaly[966]: time="2026-09-13T10:17:45.297Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4929ZQRKFKW6TJVV0RNR0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-13T10:17:55.026" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:17:45.286" grpc.time_ms=11.429 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2045gitlab # [ 423.532805] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.126Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c82363e5a93a803cfcfa1edc","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c82363e5a93a803cfcfa1edc: done: 0.015452 sec","job_status":"done","queue_duration_s":101.782292,"scheduling_latency_s":101.782308,"redis_calls":13,"redis_duration_s":0.003114,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000945,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002169,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009401,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015452,"completed_at":"2026-09-13T10:17:44.125Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2046gitlab # [ 423.571347] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.127Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8cbffbeb2a20db461bb77583","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8cbffbeb2a20db461bb77583: start","job_status":"start","queue_duration_s":101.799228,"scheduling_latency_s":101.799244}2047gitlab # [ 423.583671] gitaly[966]: time="2026-09-13T10:17:45.387Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=5 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D4929ZQRKFKW6TJVV0RNR0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-13T10:17:55.107" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:17:45.367" grpc.time_ms=20.214 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2048gitlab # [ 423.596775] gitaly[966]: time="2026-09-13T10:17:45.470Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D4929ZQRKFKW6TJVV0RNR0 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3937 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:17:45.468" grpc.time_ms=1.229 pid=966 span.kind=server system=grpc2049gitlab # [ 423.606207] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.142Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8cbffbeb2a20db461bb77583","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8cbffbeb2a20db461bb77583: done: 0.015098 sec","job_status":"done","queue_duration_s":101.799228,"scheduling_latency_s":101.799244,"redis_calls":13,"redis_duration_s":0.002709,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00093,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001779,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009484,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015098,"completed_at":"2026-09-13T10:17:44.142Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2050gitlab # [ 423.645008] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.144Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0fdb89996c426c5d3eb1afe9","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0fdb89996c426c5d3eb1afe9: start","job_status":"start","queue_duration_s":101.815743,"scheduling_latency_s":101.815759}2051gitlab # [ 423.657184] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.158Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0fdb89996c426c5d3eb1afe9","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0fdb89996c426c5d3eb1afe9: done: 0.01445 sec","job_status":"done","queue_duration_s":101.815743,"scheduling_latency_s":101.815759,"redis_calls":13,"redis_duration_s":0.002414,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0009,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001514,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009235,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01445,"completed_at":"2026-09-13T10:17:44.158Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2052gitlab # [ 423.695455] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.159Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c3dc485dc7e1d521ceb49179","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c3dc485dc7e1d521ceb49179: start","job_status":"start","queue_duration_s":101.831616,"scheduling_latency_s":101.831631}2053gitlab # [ 423.708629] gitaly[966]: time="2026-09-13T10:17:45.517Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D4929ZQRKFKW6TJVV0RNR0 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3939 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:17:45.514" grpc.time_ms=2.161 pid=966 span.kind=server system=grpc2054gitlab # [ 423.717894] gitaly[966]: time="2026-09-13T10:17:45.529Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2D4929ZQRKFKW6TJVV0RNR0 diskcache=13001c16-e938-4e7c-b893-0411e775ca74 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-13T10:17:55.150" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:17:45.410" pid=966 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root2055gitlab # [ 423.728005] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.174Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c3dc485dc7e1d521ceb49179","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c3dc485dc7e1d521ceb49179: done: 0.014786 sec","job_status":"done","queue_duration_s":101.831616,"scheduling_latency_s":101.831631,"redis_calls":13,"redis_duration_s":0.00262,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000923,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001697,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009457,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014786,"completed_at":"2026-09-13T10:17:44.174Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2056gitlab # [ 423.766122] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.176Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bfaed11afe377ec327c1e375","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bfaed11afe377ec327c1e375: start","job_status":"start","queue_duration_s":101.847929,"scheduling_latency_s":101.847945}2057gitlab # [ 423.778186] gitaly[966]: time="2026-09-13T10:17:45.529Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=126 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5063 command.oublock=8 command.real_time_ms=111 command.spawn_token_fork_ms=1 command.system_time_ms=75 command.user_time_ms=51 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4929ZQRKFKW6TJVV0RNR0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-13T10:17:55.150" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:17:45.410" grpc.time_ms=118.555 pid=966 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root2058gitlab # [ 423.791794] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.190Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bfaed11afe377ec327c1e375","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bfaed11afe377ec327c1e375: done: 0.014614 sec","job_status":"done","queue_duration_s":101.847929,"scheduling_latency_s":101.847945,"redis_calls":13,"redis_duration_s":0.002486,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00091,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001576,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00931,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014614,"completed_at":"2026-09-13T10:17:44.190Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2059gitlab # [ 423.830620] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.192Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"419cb764ca4ff761fb05c125","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-419cb764ca4ff761fb05c125: start","job_status":"start","queue_duration_s":101.863961,"scheduling_latency_s":101.863976}2060gitlab # [ 423.842943] gitaly[966]: time="2026-09-13T10:17:45.554Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2D4929ZQRKFKW6TJVV0RNR0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T10:17:55.293" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:17:45.553" grpc.time_ms=0.295 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2061gitlab # [ 423.853481] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.207Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"419cb764ca4ff761fb05c125","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-419cb764ca4ff761fb05c125: done: 0.015102 sec","job_status":"done","queue_duration_s":101.863961,"scheduling_latency_s":101.863976,"redis_calls":13,"redis_duration_s":0.002855,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000954,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001901,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009456,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015102,"completed_at":"2026-09-13T10:17:44.207Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2062gitlab # [ 423.891843] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.208Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6953b5227ee6d280485c4937","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6953b5227ee6d280485c4937: start","job_status":"start","queue_duration_s":101.88043,"scheduling_latency_s":101.880446}2063gitlab # [ 423.904190] gitaly[966]: time="2026-09-13T10:17:45.583Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=860 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=2 command.system_time_ms=12 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4929ZQRKFKW6TJVV0RNR0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-13T10:17:55.299" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:17:45.560" grpc.time_ms=23.543 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2064gitlab # [ 423.918002] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.223Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6953b5227ee6d280485c4937","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6953b5227ee6d280485c4937: done: 0.015146 sec","job_status":"done","queue_duration_s":101.88043,"scheduling_latency_s":101.880446,"redis_calls":13,"redis_duration_s":0.0030889999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001246,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001843,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009143,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015146,"completed_at":"2026-09-13T10:17:44.223Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2065gitlab # [ 423.956378] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.225Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"90ec24e06f4321c9d0bf0b75","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-90ec24e06f4321c9d0bf0b75: start","job_status":"start","queue_duration_s":101.896945,"scheduling_latency_s":101.89696}2066gitlab # [ 423.968413] gitaly[966]: time="2026-09-13T10:17:45.733Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=313 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D4929ZQRKFKW6TJVV0RNR0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-09-13T10:18:15.020" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:17:45.721" grpc.time_ms=12.074 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2067gitlab # [ 423.983127] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.239Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"90ec24e06f4321c9d0bf0b75","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-90ec24e06f4321c9d0bf0b75: done: 0.01476 sec","job_status":"done","queue_duration_s":101.896945,"scheduling_latency_s":101.89696,"redis_calls":13,"redis_duration_s":0.002701,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000901,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0018,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009103,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01476,"completed_at":"2026-09-13T10:17:44.239Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2068gitlab # [ 424.021450] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.241Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"00b44721f89d59518517ec4b","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-00b44721f89d59518517ec4b: start","job_status":"start","queue_duration_s":101.913009,"scheduling_latency_s":101.913025}2069gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=456921b37007e803dac208d4de47f6e13ba8b9e4'2070gitlab # [ 424.033555] gitaly[966]: time="2026-09-13T10:17:45.784Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2D4929ZQRKFKW6TJVV0RNR0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-13T10:17:55.003" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:17:45.784" grpc.time_ms=0.506 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2071gitlab # [ 424.043838] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.256Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"00b44721f89d59518517ec4b","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-00b44721f89d59518517ec4b: done: 0.014818 sec","job_status":"done","queue_duration_s":101.913009,"scheduling_latency_s":101.913025,"redis_calls":13,"redis_duration_s":0.002632,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000958,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001674,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009131,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014818,"completed_at":"2026-09-13T10:17:44.256Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2072gitlab # [ 424.079536] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.257Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0b7bb8f9b5525434a4fc3e60","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0b7bb8f9b5525434a4fc3e60: start","job_status":"start","queue_duration_s":101.929134,"scheduling_latency_s":101.929149}2073gitlab # [ 424.091631] gitaly[966]: time="2026-09-13T10:17:45.799Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=315 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D4929ZQRKFKW6TJVV0RNR0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-13T10:17:55.007" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-09-13T10:17:45.788" grpc.time_ms=10.999 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2074gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=456921b37007e803dac208d4de47f6e13ba8b9e4', in 0.10 seconds)2075gitlab # [ 424.104856] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.271Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0b7bb8f9b5525434a4fc3e60","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0b7bb8f9b5525434a4fc3e60: done: 0.014511 sec","job_status":"done","queue_duration_s":101.929134,"scheduling_latency_s":101.929149,"redis_calls":13,"redis_duration_s":0.002503,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000921,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001582,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009105,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014511,"completed_at":"2026-09-13T10:17:44.271Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2076gitlab # [ 424.144291] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.273Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf2c231175291ec23441eaeb","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf2c231175291ec23441eaeb: start","job_status":"start","queue_duration_s":101.945047,"scheduling_latency_s":101.945064}2077gitlab # [ 424.156259] gitaly[966]: time="2026-09-13T10:17:45.838Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2D4929ZQRKFKW6TJVV0RNR0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:17:55.041" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:17:45.821" grpc.time_ms=16.312 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2078gitlab # [ 424.169476] nixbot[3628]: INFO: ::ffff:127.0.0.1:35078 - "GET /api/repos/gitlab/root/test-flake/builds?commit=456921b37007e803dac208d4de47f6e13ba8b9e4 HTTP/1.1" 200 OK2079gitlab # [ 424.171384] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.288Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf2c231175291ec23441eaeb","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf2c231175291ec23441eaeb: done: 0.014751 sec","job_status":"done","queue_duration_s":101.945047,"scheduling_latency_s":101.945064,"redis_calls":13,"redis_duration_s":0.002638,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000935,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001703,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0092,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014751,"completed_at":"2026-09-13T10:17:44.288Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2080gitlab # [ 424.209684] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.289Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0b7fa7f12698ba2c22ac3848","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0b7fa7f12698ba2c22ac3848: start","job_status":"start","queue_duration_s":101.961159,"scheduling_latency_s":101.961176}2081gitlab # [ 424.221740] gitaly[966]: time="2026-09-13T10:17:45.863Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01M2D4929ZQRKFKW6TJVV0RNR0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:17:55.068" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:17:45.848" grpc.time_ms=15.448 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2082gitlab # [ 424.234742] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.304Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0b7fa7f12698ba2c22ac3848","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0b7fa7f12698ba2c22ac3848: done: 0.014819 sec","job_status":"done","queue_duration_s":101.961159,"scheduling_latency_s":101.961176,"redis_calls":13,"redis_duration_s":0.002677,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000935,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001742,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009307,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014819,"completed_at":"2026-09-13T10:17:44.304Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2083gitlab # [ 424.273384] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.305Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"796723c20ebf1559292decbb","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-796723c20ebf1559292decbb: start","job_status":"start","queue_duration_s":101.97746,"scheduling_latency_s":101.977477}2084gitlab # [ 424.285469] gitaly[966]: time="2026-09-13T10:17:45.914Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=21 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=882 command.oublock=0 command.real_time_ms=22 command.spawn_token_fork_ms=3 command.system_time_ms=18 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2D4929ZQRKFKW6TJVV0RNR0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-13T10:18:15.185" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:17:45.886" grpc.time_ms=27.777 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2085gitlab # [ 424.300363] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.320Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"796723c20ebf1559292decbb","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-796723c20ebf1559292decbb: done: 0.014619 sec","job_status":"done","queue_duration_s":101.97746,"scheduling_latency_s":101.977477,"redis_calls":13,"redis_duration_s":0.002619,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00089,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001729,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009182,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014619,"completed_at":"2026-09-13T10:17:44.320Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2086gitlab # [ 424.340086] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.321Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1a8f6244fea59e601b7e2730","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1a8f6244fea59e601b7e2730: start","job_status":"start","queue_duration_s":101.993396,"scheduling_latency_s":101.993413}2087gitlab # [ 424.352697] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.336Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1a8f6244fea59e601b7e2730","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1a8f6244fea59e601b7e2730: done: 0.014698 sec","job_status":"done","queue_duration_s":101.993396,"scheduling_latency_s":101.993413,"redis_calls":13,"redis_duration_s":0.002637,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000933,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001704,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009324,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014698,"completed_at":"2026-09-13T10:17:44.336Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2088gitlab # [ 424.391050] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.337Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b2362a95c1c724e838308d0b","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b2362a95c1c724e838308d0b: start","job_status":"start","queue_duration_s":102.009595,"scheduling_latency_s":102.00961}2089gitlab # [ 424.403464] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.352Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b2362a95c1c724e838308d0b","created_at":"2026-09-13T10:16:02.192Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-d71d8cffb2424736","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c833c23fa5393f82d9a7f493e2c430b8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.328Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b2362a95c1c724e838308d0b: done: 0.014641 sec","job_status":"done","queue_duration_s":102.009595,"scheduling_latency_s":102.00961,"redis_calls":13,"redis_duration_s":0.002627,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000894,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001733,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009209,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014641,"completed_at":"2026-09-13T10:17:44.352Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2090gitlab # [ 424.441974] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.353Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9a21ff3ad1d71fd5f85a3631","created_at":"2026-09-13T10:16:02.370Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-2b355fd6bccc42eb","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8579cd00cd8da0c4b0cd2058d6e29b28","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.394Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9a21ff3ad1d71fd5f85a3631: start","job_status":"start","queue_duration_s":101.95895,"scheduling_latency_s":101.958968}2091gitlab # [ 424.454426] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.376Z","message":"No existing merge request to be cleaned up."}2092gitlab # [ 424.456069] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.390Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9a21ff3ad1d71fd5f85a3631","created_at":"2026-09-13T10:16:02.370Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-2b355fd6bccc42eb","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8579cd00cd8da0c4b0cd2058d6e29b28","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.394Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9a21ff3ad1d71fd5f85a3631: done: 0.036742 sec","job_status":"done","queue_duration_s":101.95895,"scheduling_latency_s":101.958968,"redis_calls":14,"redis_duration_s":0.003927,"redis_read_bytes":12,"redis_write_bytes":1678,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001113,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":656,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001086,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001728,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":5,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021899,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036742,"completed_at":"2026-09-13T10:17:44.390Z","load_balancing_strategy":"primary","db_duration_s":0.007671,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2093gitlab # [ 424.495428] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.392Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c6eb3f6977aaeb161b8ee8c7","created_at":"2026-09-13T10:16:02.370Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-2b355fd6bccc42eb","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8579cd00cd8da0c4b0cd2058d6e29b28","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.394Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c6eb3f6977aaeb161b8ee8c7: start","job_status":"start","queue_duration_s":101.997195,"scheduling_latency_s":101.997215}2094gitlab # [ 424.507928] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.403Z","message":"No existing merge request to be cleaned up."}2095gitlab # [ 424.509692] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.411Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c6eb3f6977aaeb161b8ee8c7","created_at":"2026-09-13T10:16:02.370Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-2b355fd6bccc42eb","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8579cd00cd8da0c4b0cd2058d6e29b28","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.394Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c6eb3f6977aaeb161b8ee8c7: done: 0.019841 sec","job_status":"done","queue_duration_s":101.997195,"scheduling_latency_s":101.997215,"redis_calls":10,"redis_duration_s":0.002511,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001051,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00146,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012524,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019841,"completed_at":"2026-09-13T10:17:44.411Z","load_balancing_strategy":"primary","db_duration_s":0.002631,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2096gitlab # [ 424.548382] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.413Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"5dab4241385bc1eebd015ffd","created_at":"2026-09-13T10:16:02.370Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-2b355fd6bccc42eb","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8579cd00cd8da0c4b0cd2058d6e29b28","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.394Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-5dab4241385bc1eebd015ffd: start","job_status":"start","queue_duration_s":102.018389,"scheduling_latency_s":102.018406}2097gitlab # [ 424.560390] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.424Z","message":"No existing merge request to be cleaned up."}2098gitlab # [ 424.562048] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.432Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"5dab4241385bc1eebd015ffd","created_at":"2026-09-13T10:16:02.370Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-2b355fd6bccc42eb","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8579cd00cd8da0c4b0cd2058d6e29b28","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.394Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-5dab4241385bc1eebd015ffd: done: 0.019399 sec","job_status":"done","queue_duration_s":102.018389,"scheduling_latency_s":102.018406,"redis_calls":10,"redis_duration_s":0.002423,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001004,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001419,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012452,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019399,"completed_at":"2026-09-13T10:17:44.432Z","load_balancing_strategy":"primary","db_duration_s":0.002498,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2099gitlab # [ 424.600108] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.434Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"1cc0e3358328b45761fe7801","created_at":"2026-09-13T10:16:02.370Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-2b355fd6bccc42eb","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8579cd00cd8da0c4b0cd2058d6e29b28","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.394Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-1cc0e3358328b45761fe7801: start","job_status":"start","queue_duration_s":102.039326,"scheduling_latency_s":102.039343}2100gitlab # [ 424.611936] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.445Z","message":"No existing merge request to be cleaned up."}2101gitlab # [ 424.613710] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.453Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"1cc0e3358328b45761fe7801","created_at":"2026-09-13T10:16:02.370Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-2b355fd6bccc42eb","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8579cd00cd8da0c4b0cd2058d6e29b28","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:02.394Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-1cc0e3358328b45761fe7801: done: 0.019096 sec","job_status":"done","queue_duration_s":102.039326,"scheduling_latency_s":102.039343,"redis_calls":10,"redis_duration_s":0.002338,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000953,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001385,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012368,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019096,"completed_at":"2026-09-13T10:17:44.453Z","load_balancing_strategy":"primary","db_duration_s":0.002165,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2102gitlab # [ 424.651356] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.454Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"800b2b5868c8fe3ea389f885","created_at":"2026-09-13T10:16:09.404Z","trace_propagation_headers":{"sentry-trace":"efc149dd85694fe59bcf8a3fe85d4e0a-5160422b0f454b4b","baggage":"sentry-trace_id=efc149dd85694fe59bcf8a3fe85d4e0a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D48RBD91NEYGQF89A3QRJ6","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:09.469Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-800b2b5868c8fe3ea389f885: start","job_status":"start","queue_duration_s":94.985693,"scheduling_latency_s":94.985709}2103gitlab # [ 424.665930] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.589Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"800b2b5868c8fe3ea389f885","created_at":"2026-09-13T10:16:09.404Z","trace_propagation_headers":{"sentry-trace":"efc149dd85694fe59bcf8a3fe85d4e0a-5160422b0f454b4b","baggage":"sentry-trace_id=efc149dd85694fe59bcf8a3fe85d4e0a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D48RBD91NEYGQF89A3QRJ6","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:09.469Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-800b2b5868c8fe3ea389f885: done: 0.13442 sec","job_status":"done","queue_duration_s":94.985693,"scheduling_latency_s":94.985709,"redis_calls":13,"redis_duration_s":0.00383,"redis_read_bytes":138,"redis_write_bytes":1593,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000763,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":428,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001786,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001281,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.077892,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.09947442,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.13442,"completed_at":"2026-09-13T10:17:44.589Z","load_balancing_strategy":"primary","db_duration_s":0.050912,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2104gitlab # [ 424.709500] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.591Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"481f5f371a5564d33df52711","created_at":"2026-09-13T10:16:09.489Z","trace_propagation_headers":{"sentry-trace":"efc149dd85694fe59bcf8a3fe85d4e0a-5160422b0f454b4b","baggage":"sentry-trace_id=efc149dd85694fe59bcf8a3fe85d4e0a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D48RBD91NEYGQF89A3QRJ6","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:09.498Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-481f5f371a5564d33df52711: start","job_status":"start","queue_duration_s":95.092799,"scheduling_latency_s":95.09282}2105gitlab # [ 424.723994] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.714Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"481f5f371a5564d33df52711","created_at":"2026-09-13T10:16:09.489Z","trace_propagation_headers":{"sentry-trace":"efc149dd85694fe59bcf8a3fe85d4e0a-5160422b0f454b4b","baggage":"sentry-trace_id=efc149dd85694fe59bcf8a3fe85d4e0a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D48RBD91NEYGQF89A3QRJ6","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:09.498Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-481f5f371a5564d33df52711: done: 0.123272 sec","job_status":"done","queue_duration_s":95.092799,"scheduling_latency_s":95.09282,"redis_calls":14,"redis_duration_s":0.003939,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000757,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001868,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001314,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.082373,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.09636644,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.123272,"completed_at":"2026-09-13T10:17:44.714Z","load_balancing_strategy":"primary","db_duration_s":0.033786,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2106gitlab # [ 424.769878] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.716Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"98b0acd25043694044a0f43f","created_at":"2026-09-13T10:16:10.356Z","trace_propagation_headers":{"sentry-trace":"efc149dd85694fe59bcf8a3fe85d4e0a-5160422b0f454b4b","baggage":"sentry-trace_id=efc149dd85694fe59bcf8a3fe85d4e0a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D48RBD91NEYGQF89A3QRJ6","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:10.363Z","job_size_bytes":5,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-98b0acd25043694044a0f43f: start","job_status":"start","queue_duration_s":94.352563,"scheduling_latency_s":94.352586}2107gitlab # [ 424.819522] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.904Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"98b0acd25043694044a0f43f","created_at":"2026-09-13T10:16:10.356Z","trace_propagation_headers":{"sentry-trace":"efc149dd85694fe59bcf8a3fe85d4e0a-5160422b0f454b4b","baggage":"sentry-trace_id=efc149dd85694fe59bcf8a3fe85d4e0a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D48RBD91NEYGQF89A3QRJ6","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:10.363Z","job_size_bytes":5,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-98b0acd25043694044a0f43f: done: 0.187963 sec","job_status":"done","queue_duration_s":94.352563,"scheduling_latency_s":94.352586,"redis_calls":12,"redis_duration_s":0.00654,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000828,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.005258,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000454,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.139494,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00027904,"duration_s":0.187963,"completed_at":"2026-09-13T10:17:44.904Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.039716,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2108gitlab # [ 424.910966] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:44.905Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"a82e5b7d0937e57fc90eb1db","created_at":"2026-09-13T10:16:10.397Z","trace_propagation_headers":{"sentry-trace":"efc149dd85694fe59bcf8a3fe85d4e0a-5160422b0f454b4b","baggage":"sentry-trace_id=efc149dd85694fe59bcf8a3fe85d4e0a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D48RBD91NEYGQF89A3QRJ6","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:10.402Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-a82e5b7d0937e57fc90eb1db: start","job_status":"start","queue_duration_s":94.502971,"scheduling_latency_s":94.502989}2109gitlab # [ 424.933626] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:45.167Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"a82e5b7d0937e57fc90eb1db","created_at":"2026-09-13T10:16:10.397Z","trace_propagation_headers":{"sentry-trace":"efc149dd85694fe59bcf8a3fe85d4e0a-5160422b0f454b4b","baggage":"sentry-trace_id=efc149dd85694fe59bcf8a3fe85d4e0a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D48RBD91NEYGQF89A3QRJ6","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:10.402Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-a82e5b7d0937e57fc90eb1db: done: 0.261993 sec","job_status":"done","queue_duration_s":94.502971,"scheduling_latency_s":94.502989,"redis_calls":11,"redis_duration_s":0.0030989999999999998,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000931,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001548,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00062,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.042,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.222,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.222,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.041,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.157226,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00055516,"duration_s":0.261993,"completed_at":"2026-09-13T10:17:45.167Z","load_balancing_strategy":"primary","db_duration_s":0.108832,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2110gitlab # [ 424.985737] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:45.169Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"e615139dbd7481f0575da12f","created_at":"2026-09-13T10:16:10.405Z","trace_propagation_headers":{"sentry-trace":"efc149dd85694fe59bcf8a3fe85d4e0a-5160422b0f454b4b","baggage":"sentry-trace_id=efc149dd85694fe59bcf8a3fe85d4e0a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D48RBD91NEYGQF89A3QRJ6","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:10.410Z","job_size_bytes":89,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-e615139dbd7481f0575da12f: start","job_status":"start","queue_duration_s":94.758639,"scheduling_latency_s":94.75866}2111gitlab # [ 425.033237] gitaly[966]: time="2026-09-13T10:17:46.742Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D4929ZQRKFKW6TJVV0RNR0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-13T10:17:56.010" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:17:46.740" grpc.time_ms=2.03 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2112gitlab # [ 425.063152] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:45.213Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"e615139dbd7481f0575da12f","created_at":"2026-09-13T10:16:10.405Z","trace_propagation_headers":{"sentry-trace":"efc149dd85694fe59bcf8a3fe85d4e0a-5160422b0f454b4b","baggage":"sentry-trace_id=efc149dd85694fe59bcf8a3fe85d4e0a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D48RBD91NEYGQF89A3QRJ6","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:10.410Z","job_size_bytes":89,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-e615139dbd7481f0575da12f: done: 0.043528 sec","job_status":"done","queue_duration_s":94.758639,"scheduling_latency_s":94.75866,"redis_calls":11,"redis_duration_s":0.002912,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000721,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001618,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000573,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027409,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00040302,"duration_s":0.043528,"completed_at":"2026-09-13T10:17:45.213Z","load_balancing_strategy":"primary","db_duration_s":0.007972,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2113gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=456921b37007e803dac208d4de47f6e13ba8b9e4'2114gitlab # [ 425.133885] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:45.214Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 a97336d9b6bb4bb908a8849d973a90d364eab2ac refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"df46576d4223fd33eb04ddd3","created_at":"2026-09-13T10:16:19.004Z","trace_propagation_headers":{"sentry-trace":"c9b02740ec404b059c9837f1b36e1571-0fd4a9c40a21456c","baggage":"sentry-trace_id=c9b02740ec404b059c9837f1b36e1571,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D4929ZQRKFKW6TJVV0RNR0","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:19.046Z","job_size_bytes":185,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-df46576d4223fd33eb04ddd3: start","job_status":"start","queue_duration_s":86.168194,"scheduling_latency_s":86.168214}2115gitlab # [ 425.151069] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:45.233Z","message":"changes: \"0000000000000000000000000000000000000000 a97336d9b6bb4bb908a8849d973a90d364eab2ac refs/heads/master\\n\""}2116gitlab # [ 425.154414] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:46.778Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 a97336d9b6bb4bb908a8849d973a90d364eab2ac refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"df46576d4223fd33eb04ddd3","created_at":"2026-09-13T10:16:19.004Z","trace_propagation_headers":{"sentry-trace":"c9b02740ec404b059c9837f1b36e1571-0fd4a9c40a21456c","baggage":"sentry-trace_id=c9b02740ec404b059c9837f1b36e1571,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D4929ZQRKFKW6TJVV0RNR0","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:16:19.046Z","job_size_bytes":185,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-df46576d4223fd33eb04ddd3: done: 1.563971 sec","job_status":"done","queue_duration_s":86.168194,"scheduling_latency_s":86.168214,"gitaly_calls":13,"gitaly_duration_s":0.324229,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.218141,"redis_read_bytes":2108,"redis_write_bytes":21751,"redis_cache_calls":15,"redis_cache_duration_s":0.006106,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.008509,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.002318,"redis_queues_read_bytes":15,"redis_queues_write_bytes":7839,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.004919,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.025444,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.170845,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.087,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.488,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.531,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.087,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.940722,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00077858,"exclusive_lock_hold_duration_s":0.03460756,"duration_s":1.563971,"completed_at":"2026-09-13T10:17:46.778Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.154823,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2117gitlab # [ 425.204456] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:46.781Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"e062d889e3beba40210ddd5e","created_at":"2026-09-13T10:16:25.382Z","trace_propagation_headers":{"sentry-trace":"b857f0cecad048e38efbe21aef5b640d-f920244e49aa4c12","baggage":"sentry-trace_id=b857f0cecad048e38efbe21aef5b640d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D498P87CGPP5YH2KQ33ZAW","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:25.392Z","job_size_bytes":65,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-e062d889e3beba40210ddd5e: start","job_status":"start","queue_duration_s":81.387587,"scheduling_latency_s":81.388215}2118gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=456921b37007e803dac208d4de47f6e13ba8b9e4', in 0.09 seconds)2119gitlab # [ 425.220596] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:46.880Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"e062d889e3beba40210ddd5e","created_at":"2026-09-13T10:16:25.382Z","trace_propagation_headers":{"sentry-trace":"b857f0cecad048e38efbe21aef5b640d-f920244e49aa4c12","baggage":"sentry-trace_id=b857f0cecad048e38efbe21aef5b640d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D498P87CGPP5YH2KQ33ZAW","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:25.392Z","job_size_bytes":65,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-e062d889e3beba40210ddd5e: done: 0.100313 sec","job_status":"done","queue_duration_s":81.387587,"scheduling_latency_s":81.388215,"redis_calls":16,"redis_duration_s":0.004508,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.002392,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001564,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000552,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.058996,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00050216,"duration_s":0.100313,"completed_at":"2026-09-13T10:17:46.880Z","load_balancing_strategy":"primary","db_duration_s":0.033513,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2120gitlab # [ 425.264251] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:46.882Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"209b700e70cf4eb2ec932b82","created_at":"2026-09-13T10:16:25.395Z","trace_propagation_headers":{"sentry-trace":"b857f0cecad048e38efbe21aef5b640d-f920244e49aa4c12","baggage":"sentry-trace_id=b857f0cecad048e38efbe21aef5b640d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D498P87CGPP5YH2KQ33ZAW","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:25.403Z","job_size_bytes":65,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-209b700e70cf4eb2ec932b82: start","job_status":"start","queue_duration_s":81.479203,"scheduling_latency_s":81.479223}2121gitlab # [ 425.281419] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:47.002Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"209b700e70cf4eb2ec932b82","created_at":"2026-09-13T10:16:25.395Z","trace_propagation_headers":{"sentry-trace":"b857f0cecad048e38efbe21aef5b640d-f920244e49aa4c12","baggage":"sentry-trace_id=b857f0cecad048e38efbe21aef5b640d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D498P87CGPP5YH2KQ33ZAW","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:25.403Z","job_size_bytes":65,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-209b700e70cf4eb2ec932b82: done: 0.120061 sec","job_status":"done","queue_duration_s":81.479203,"scheduling_latency_s":81.479223,"redis_calls":15,"redis_duration_s":0.004528,"redis_read_bytes":142,"redis_write_bytes":1431,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000863,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":431,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001829,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001836,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.033,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.067735,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00053444,"duration_s":0.120061,"completed_at":"2026-09-13T10:17:47.002Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.04088,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2122gitlab # [ 425.326090] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:47.004Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"436d1c7f1eb47a588710855f","created_at":"2026-09-13T10:16:25.590Z","trace_propagation_headers":{"sentry-trace":"b857f0cecad048e38efbe21aef5b640d-f920244e49aa4c12","baggage":"sentry-trace_id=b857f0cecad048e38efbe21aef5b640d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D498P87CGPP5YH2KQ33ZAW","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:25.606Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-436d1c7f1eb47a588710855f: start","job_status":"start","queue_duration_s":81.398143,"scheduling_latency_s":81.398168}2123gitlab # [ 425.342114] nixbot[3628]: INFO: ::ffff:127.0.0.1:35092 - "GET /api/repos/gitlab/root/test-flake/builds?commit=456921b37007e803dac208d4de47f6e13ba8b9e4 HTTP/1.1" 200 OK2124gitlab # [ 425.352049] gitaly[966]: time="2026-09-13T10:17:47.261Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D498P87CGPP5YH2KQ33ZAW grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:17:47.260" grpc.time_ms=0.816 pid=966 span.kind=server system=grpc2125gitlab # [ 425.393990] gitaly[966]: time="2026-09-13T10:17:47.303Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D498P87CGPP5YH2KQ33ZAW grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:17:47.301" grpc.time_ms=1.301 pid=966 span.kind=server system=grpc2126gitlab # [ 425.419664] gitaly[966]: time="2026-09-13T10:17:47.328Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D498P87CGPP5YH2KQ33ZAW grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:17:47.327" grpc.time_ms=0.608 pid=966 span.kind=server system=grpc2127gitlab # [ 425.436827] gitaly[966]: time="2026-09-13T10:17:47.341Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2D498P87CGPP5YH2KQ33ZAW diskcache=58833b5c-a4e2-4de3-9fe6-423a28a1d344 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T10:18:17.001" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:17:47.202" pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2128gitlab # [ 425.452858] gitaly[966]: time="2026-09-13T10:17:47.341Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=155 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7574 command.oublock=0 command.real_time_ms=132 command.spawn_token_fork_ms=1 command.system_time_ms=108 command.user_time_ms=47 component=gitaly.UnaryServerInterceptor correlation_id=01M2D498P87CGPP5YH2KQ33ZAW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T10:18:17.001" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:17:47.202" grpc.time_ms=138.845 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2129gitlab # [ 425.637276] gitaly[966]: time="2026-09-13T10:17:47.546Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D498P87CGPP5YH2KQ33ZAW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-13T10:18:17.344" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:17:47.544" grpc.time_ms=1.547 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2130gitlab # [ 425.724536] gitaly[966]: time="2026-09-13T10:17:47.633Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D498P87CGPP5YH2KQ33ZAW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:18:17.431" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:17:47.631" grpc.time_ms=1.441 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2131gitlab # [ 425.769866] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:47.677Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"436d1c7f1eb47a588710855f","created_at":"2026-09-13T10:16:25.590Z","trace_propagation_headers":{"sentry-trace":"b857f0cecad048e38efbe21aef5b640d-f920244e49aa4c12","baggage":"sentry-trace_id=b857f0cecad048e38efbe21aef5b640d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D498P87CGPP5YH2KQ33ZAW","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-13T10:16:25.606Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-436d1c7f1eb47a588710855f: done: 0.673316 sec","job_status":"done","queue_duration_s":81.398143,"scheduling_latency_s":81.398168,"gitaly_calls":3,"gitaly_duration_s":0.152981,"redis_calls":117,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.040679,"redis_read_bytes":458,"redis_write_bytes":28002,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001783,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.003498,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.012845,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":25,"redis_queues_duration_s":0.005321,"redis_queues_read_bytes":37,"redis_queues_write_bytes":14496,"redis_queues_metadata_calls":21,"redis_queues_metadata_duration_s":0.010745,"redis_queues_metadata_read_bytes":32,"redis_queues_metadata_write_bytes":3370,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000786,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000857,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003061,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.095,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.034,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.04,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.046,"db_ci_duration_s":0.049,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.392998,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.57323222,"duration_s":0.673316,"completed_at":"2026-09-13T10:17:47.677Z","load_balancing_strategy":"primary","db_duration_s":0.112793,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2132gitlab # [ 425.876577] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:47.681Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","a97336d9b6bb4bb908a8849d973a90d364eab2ac 456921b37007e803dac208d4de47f6e13ba8b9e4 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"2028c93f423f4c7ea2b886f3","created_at":"2026-09-13T10:17:41.421Z","trace_propagation_headers":{"sentry-trace":"bb684fb7a0c4498faf7f0ac698ac4cca-37d67a3e82f84199","baggage":"sentry-trace_id=bb684fb7a0c4498faf7f0ac698ac4cca,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D4BJPHAMGG58F3SJZ3VXBD","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:41.427Z","job_size_bytes":185,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-2028c93f423f4c7ea2b886f3: start","job_status":"start","queue_duration_s":6.253285,"scheduling_latency_s":6.253371}2133gitlab # [ 425.904267] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:47.703Z","message":"changes: \"a97336d9b6bb4bb908a8849d973a90d364eab2ac 456921b37007e803dac208d4de47f6e13ba8b9e4 refs/heads/master\\n\""}2134gitlab # [ 425.907623] gitaly[966]: time="2026-09-13T10:17:47.755Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D4BJPHAMGG58F3SJZ3VXBD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:17:57.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:17:47.736" grpc.time_ms=18.641 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2135gitlab # [ 425.924398] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:47.784Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"08bd1ecd64a8ad7adbae34a1","created_at":"2026-09-13T10:17:47.780Z","trace_propagation_headers":{"sentry-trace":"bb684fb7a0c4498faf7f0ac698ac4cca-a5f8c8240c084193","baggage":"sentry-trace_id=bb684fb7a0c4498faf7f0ac698ac4cca,sentry-release=Unknown"},"correlation_id":"01M2D4BJPHAMGG58F3SJZ3VXBD","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"2359f465cea068c7491854ba","job_size_bytes":46,"pid":3504,"job_status":"deduplicated","message":"ProjectCacheWorker JID-08bd1ecd64a8ad7adbae34a1: deduplicated: until executing","deduplication.type":"until executing"}2136gitlab # [ 425.940297] gitaly[966]: time="2026-09-13T10:17:47.776Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4BJPHAMGG58F3SJZ3VXBD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-13T10:17:57.032" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:17:47.762" grpc.time_ms=13.14 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2137gitlab # [ 425.953598] gitaly[966]: time="2026-09-13T10:17:47.830Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=4 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D4BJPHAMGG58F3SJZ3VXBD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-13T10:17:57.032" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:17:47.812" grpc.time_ms=17.167 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2138gitlab # [ 425.967808] gitaly[966]: time="2026-09-13T10:17:47.854Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=596 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2D4BJPHAMGG58F3SJZ3VXBD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-09-13T10:18:17.137" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:17:47.838" grpc.time_ms=16.73 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2139gitlab # [ 425.982933] gitaly[966]: time="2026-09-13T10:17:47.887Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=319 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D4BJPHAMGG58F3SJZ3VXBD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-13T10:17:57.095" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-09-13T10:17:47.876" grpc.time_ms=10.896 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2140gitlab # [ 426.012506] gitaly[966]: time="2026-09-13T10:17:47.921Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=1 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D4BJPHAMGG58F3SJZ3VXBD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:17:57.124" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:17:47.904" grpc.time_ms=16.667 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2141gitlab # [ 426.196212] gitaly[966]: time="2026-09-13T10:17:48.104Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D4BJPHAMGG58F3SJZ3VXBD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-13T10:17:58.001" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:17:48.101" grpc.time_ms=2.295 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2142gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=456921b37007e803dac208d4de47f6e13ba8b9e4'2143gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=456921b37007e803dac208d4de47f6e13ba8b9e4', in 0.07 seconds)2144gitlab # [ 426.244226] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.151Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","a97336d9b6bb4bb908a8849d973a90d364eab2ac 456921b37007e803dac208d4de47f6e13ba8b9e4 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"2028c93f423f4c7ea2b886f3","created_at":"2026-09-13T10:17:41.421Z","trace_propagation_headers":{"sentry-trace":"bb684fb7a0c4498faf7f0ac698ac4cca-37d67a3e82f84199","baggage":"sentry-trace_id=bb684fb7a0c4498faf7f0ac698ac4cca,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D4BJPHAMGG58F3SJZ3VXBD","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:41.427Z","job_size_bytes":185,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-2028c93f423f4c7ea2b886f3: done: 0.470786 sec","job_status":"done","queue_duration_s":6.253285,"scheduling_latency_s":6.253371,"gitaly_calls":7,"gitaly_duration_s":0.123966,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.036747999999999996,"redis_read_bytes":501,"redis_write_bytes":11942,"redis_cache_calls":9,"redis_cache_duration_s":0.00492,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.001825,"redis_queues_read_bytes":9,"redis_queues_write_bytes":5719,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.005383,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.011478,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.013142,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.044,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.044,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.044,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.044,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.299693,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0011507,"exclusive_lock_hold_duration_s":0.03282524,"duration_s":0.470786,"completed_at":"2026-09-13T10:17:48.151Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.046469,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2145gitlab # [ 426.331418] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.155Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"965a7da69fe7b0b5334f2487","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-965a7da69fe7b0b5334f2487: start","job_status":"start","queue_duration_s":5.185559,"scheduling_latency_s":5.185608}2146gitlab # [ 426.347582] nixbot[3628]: INFO: ::ffff:127.0.0.1:35108 - "GET /api/repos/gitlab/root/test-flake/builds?commit=456921b37007e803dac208d4de47f6e13ba8b9e4 HTTP/1.1" 200 OK2147gitlab # [ 426.350081] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.239Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"965a7da69fe7b0b5334f2487","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-965a7da69fe7b0b5334f2487: done: 0.084248 sec","job_status":"done","queue_duration_s":5.185559,"scheduling_latency_s":5.185608,"redis_calls":13,"redis_duration_s":0.00546,"redis_read_bytes":12,"redis_write_bytes":1300,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000686,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":289,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001965,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":395,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002809,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":616,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.019,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.019,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.049932,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00077102,"duration_s":0.084248,"completed_at":"2026-09-13T10:17:48.239Z","load_balancing_strategy":"primary","db_duration_s":0.016329,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2148gitlab # [ 426.393115] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.242Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5489ac7bb6cd25030c606e3f","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5489ac7bb6cd25030c606e3f: start","job_status":"start","queue_duration_s":5.272505,"scheduling_latency_s":5.272543}2149gitlab # [ 426.405206] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.290Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5489ac7bb6cd25030c606e3f","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5489ac7bb6cd25030c606e3f: done: 0.048487 sec","job_status":"done","queue_duration_s":5.272505,"scheduling_latency_s":5.272543,"redis_calls":10,"redis_duration_s":0.004868,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002013,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002855,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.004,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.004,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035125,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.048487,"completed_at":"2026-09-13T10:17:48.290Z","load_balancing_strategy":"primary","db_duration_s":0.004716,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2150gitlab # [ 426.443536] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.294Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"91792fa8835a3c8b908a47a1","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-91792fa8835a3c8b908a47a1: start","job_status":"start","queue_duration_s":5.324183,"scheduling_latency_s":5.324228}2151gitlab # [ 426.455679] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.335Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"91792fa8835a3c8b908a47a1","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-91792fa8835a3c8b908a47a1: done: 0.04129 sec","job_status":"done","queue_duration_s":5.324183,"scheduling_latency_s":5.324228,"redis_calls":10,"redis_duration_s":0.0034549999999999997,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001384,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002071,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.003,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028908,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.04129,"completed_at":"2026-09-13T10:17:48.335Z","load_balancing_strategy":"primary","db_duration_s":0.003299,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2152gitlab # [ 426.493813] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.337Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ebf8d51e5c1b49e80e973962","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ebf8d51e5c1b49e80e973962: start","job_status":"start","queue_duration_s":5.367844,"scheduling_latency_s":5.367879}2153gitlab # [ 426.505786] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.369Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ebf8d51e5c1b49e80e973962","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ebf8d51e5c1b49e80e973962: done: 0.031982 sec","job_status":"done","queue_duration_s":5.367844,"scheduling_latency_s":5.367879,"redis_calls":10,"redis_duration_s":0.003066,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00126,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001806,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020551,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031982,"completed_at":"2026-09-13T10:17:48.369Z","load_balancing_strategy":"primary","db_duration_s":0.002937,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2154gitlab # [ 426.543397] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.372Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"609d83ecc2e5eeeee472f8be","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-609d83ecc2e5eeeee472f8be: start","job_status":"start","queue_duration_s":5.402295,"scheduling_latency_s":5.402344}2155gitlab # [ 426.555738] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.419Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"609d83ecc2e5eeeee472f8be","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-609d83ecc2e5eeeee472f8be: done: 0.047572 sec","job_status":"done","queue_duration_s":5.402295,"scheduling_latency_s":5.402344,"redis_calls":10,"redis_duration_s":0.003938,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00172,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002218,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.003,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033735,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.047572,"completed_at":"2026-09-13T10:17:48.419Z","load_balancing_strategy":"primary","db_duration_s":0.003866,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2156gitlab # [ 426.594748] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.422Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a10e384ae378d8927c375ce3","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a10e384ae378d8927c375ce3: start","job_status":"start","queue_duration_s":5.452864,"scheduling_latency_s":5.452911}2157gitlab # [ 426.606949] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.459Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a10e384ae378d8927c375ce3","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a10e384ae378d8927c375ce3: done: 0.037143 sec","job_status":"done","queue_duration_s":5.452864,"scheduling_latency_s":5.452911,"redis_calls":10,"redis_duration_s":0.003198,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001316,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001882,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024763,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.037143,"completed_at":"2026-09-13T10:17:48.459Z","load_balancing_strategy":"primary","db_duration_s":0.003084,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2158gitlab # [ 426.644842] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.462Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e89e20da0aaea95a6820bba7","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e89e20da0aaea95a6820bba7: start","job_status":"start","queue_duration_s":5.492451,"scheduling_latency_s":5.492482}2159gitlab # [ 426.656662] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.496Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e89e20da0aaea95a6820bba7","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e89e20da0aaea95a6820bba7: done: 0.03472 sec","job_status":"done","queue_duration_s":5.492451,"scheduling_latency_s":5.492482,"redis_calls":10,"redis_duration_s":0.003577,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001582,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001995,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022068,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.03472,"completed_at":"2026-09-13T10:17:48.496Z","load_balancing_strategy":"primary","db_duration_s":0.003109,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2160gitlab # [ 426.694162] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.500Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"92088f73d9fcb93f67e82139","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-92088f73d9fcb93f67e82139: start","job_status":"start","queue_duration_s":5.530921,"scheduling_latency_s":5.530967}2161gitlab # [ 426.706379] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.548Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"92088f73d9fcb93f67e82139","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-92088f73d9fcb93f67e82139: done: 0.04744 sec","job_status":"done","queue_duration_s":5.530921,"scheduling_latency_s":5.530967,"redis_calls":10,"redis_duration_s":0.004189,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001757,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002432,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.003,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033545,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.04744,"completed_at":"2026-09-13T10:17:48.548Z","load_balancing_strategy":"primary","db_duration_s":0.003813,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2162gitlab # [ 426.744107] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.551Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a85249ac6216ede22f6cdffd","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a85249ac6216ede22f6cdffd: start","job_status":"start","queue_duration_s":5.581193,"scheduling_latency_s":5.581232}2163gitlab # [ 426.755852] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.594Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a85249ac6216ede22f6cdffd","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a85249ac6216ede22f6cdffd: done: 0.043253 sec","job_status":"done","queue_duration_s":5.581193,"scheduling_latency_s":5.581232,"redis_calls":10,"redis_duration_s":0.004769,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002361,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002408,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.003,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0301,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.043253,"completed_at":"2026-09-13T10:17:48.594Z","load_balancing_strategy":"primary","db_duration_s":0.003842,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2164gitlab # [ 426.793931] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.597Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"0eeb6104eff20645943f7b67","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-0eeb6104eff20645943f7b67: start","job_status":"start","queue_duration_s":5.628038,"scheduling_latency_s":5.628082}2165gitlab # [ 426.807220] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.644Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"0eeb6104eff20645943f7b67","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-0eeb6104eff20645943f7b67: done: 0.046418 sec","job_status":"done","queue_duration_s":5.628038,"scheduling_latency_s":5.628082,"redis_calls":10,"redis_duration_s":0.003934,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001604,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00233,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.003,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032785,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.046418,"completed_at":"2026-09-13T10:17:48.644Z","load_balancing_strategy":"primary","db_duration_s":0.004217,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2166gitlab # [ 426.845103] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.647Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"952d69b05ba6ebe2cb199912","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-952d69b05ba6ebe2cb199912: start","job_status":"start","queue_duration_s":5.677269,"scheduling_latency_s":5.677306}2167gitlab # [ 426.857079] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.685Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"952d69b05ba6ebe2cb199912","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-952d69b05ba6ebe2cb199912: done: 0.037984 sec","job_status":"done","queue_duration_s":5.677269,"scheduling_latency_s":5.677306,"redis_calls":10,"redis_duration_s":0.004399,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001861,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002538,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.003,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025644,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.037984,"completed_at":"2026-09-13T10:17:48.684Z","load_balancing_strategy":"primary","db_duration_s":0.003894,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2168gitlab # [ 426.894911] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.687Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f6622868e03d6fb7af1de7a0","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f6622868e03d6fb7af1de7a0: start","job_status":"start","queue_duration_s":5.717476,"scheduling_latency_s":5.717507}2169gitlab # [ 426.907116] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.716Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f6622868e03d6fb7af1de7a0","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f6622868e03d6fb7af1de7a0: done: 0.029484 sec","job_status":"done","queue_duration_s":5.717476,"scheduling_latency_s":5.717507,"redis_calls":10,"redis_duration_s":0.002921,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001217,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001704,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018988,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029484,"completed_at":"2026-09-13T10:17:48.716Z","load_balancing_strategy":"primary","db_duration_s":0.003039,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2170gitlab # [ 426.944991] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.718Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f1965c37b417956284cc7092","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f1965c37b417956284cc7092: start","job_status":"start","queue_duration_s":5.748859,"scheduling_latency_s":5.748885}2171gitlab # [ 426.957152] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.745Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f1965c37b417956284cc7092","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f1965c37b417956284cc7092: done: 0.026414 sec","job_status":"done","queue_duration_s":5.748859,"scheduling_latency_s":5.748885,"redis_calls":10,"redis_duration_s":0.002837,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001138,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001699,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017536,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026414,"completed_at":"2026-09-13T10:17:48.744Z","load_balancing_strategy":"primary","db_duration_s":0.002874,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2172gitlab # [ 426.994980] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.747Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c1db8550f1298ba39a47401c","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c1db8550f1298ba39a47401c: start","job_status":"start","queue_duration_s":5.77722,"scheduling_latency_s":5.777246}2173gitlab # [ 427.008468] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.770Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c1db8550f1298ba39a47401c","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c1db8550f1298ba39a47401c: done: 0.024001 sec","job_status":"done","queue_duration_s":5.77722,"scheduling_latency_s":5.777246,"redis_calls":10,"redis_duration_s":0.0025960000000000002,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001081,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001515,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015622,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024001,"completed_at":"2026-09-13T10:17:48.770Z","load_balancing_strategy":"primary","db_duration_s":0.002591,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2174gitlab # [ 427.046474] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.773Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"0f15d12ed6602f8c6d12a601","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-0f15d12ed6602f8c6d12a601: start","job_status":"start","queue_duration_s":5.803234,"scheduling_latency_s":5.803256}2175gitlab # [ 427.058667] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.795Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"0f15d12ed6602f8c6d12a601","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-0f15d12ed6602f8c6d12a601: done: 0.022781 sec","job_status":"done","queue_duration_s":5.803234,"scheduling_latency_s":5.803256,"redis_calls":10,"redis_duration_s":0.002595,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001047,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001548,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014891,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022781,"completed_at":"2026-09-13T10:17:48.795Z","load_balancing_strategy":"primary","db_duration_s":0.002548,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2176gitlab # [ 427.096486] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.797Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9866da0228ec36c4bb3918a2","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9866da0228ec36c4bb3918a2: start","job_status":"start","queue_duration_s":5.827515,"scheduling_latency_s":5.827534}2177gitlab # [ 427.108421] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.817Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9866da0228ec36c4bb3918a2","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9866da0228ec36c4bb3918a2: done: 0.020462 sec","job_status":"done","queue_duration_s":5.827515,"scheduling_latency_s":5.827534,"redis_calls":10,"redis_duration_s":0.002399,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000972,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001427,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013081,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020462,"completed_at":"2026-09-13T10:17:48.817Z","load_balancing_strategy":"primary","db_duration_s":0.002548,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2178gitlab # [ 427.146412] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.819Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5d1c2a611137ade13677f2c8","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5d1c2a611137ade13677f2c8: start","job_status":"start","queue_duration_s":5.849684,"scheduling_latency_s":5.849703}2179gitlab # [ 427.158308] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.839Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5d1c2a611137ade13677f2c8","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5d1c2a611137ade13677f2c8: done: 0.019655 sec","job_status":"done","queue_duration_s":5.849684,"scheduling_latency_s":5.849703,"redis_calls":10,"redis_duration_s":0.002314,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000945,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001369,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01278,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019655,"completed_at":"2026-09-13T10:17:48.839Z","load_balancing_strategy":"primary","db_duration_s":0.002366,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2180gitlab # [ 427.196095] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.840Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a0d15e30b713d0a6baaaa4e0","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a0d15e30b713d0a6baaaa4e0: start","job_status":"start","queue_duration_s":5.870762,"scheduling_latency_s":5.870782}2181gitlab # [ 427.208053] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.860Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a0d15e30b713d0a6baaaa4e0","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a0d15e30b713d0a6baaaa4e0: done: 0.020004 sec","job_status":"done","queue_duration_s":5.870762,"scheduling_latency_s":5.870782,"redis_calls":10,"redis_duration_s":0.002444,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001039,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001405,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012881,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020004,"completed_at":"2026-09-13T10:17:48.860Z","load_balancing_strategy":"primary","db_duration_s":0.002379,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2182gitlab # [ 427.247465] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.862Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"91c5e53b1102f9b7b112d379","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-91c5e53b1102f9b7b112d379: start","job_status":"start","queue_duration_s":5.892774,"scheduling_latency_s":5.892793}2183gitlab # [ 427.259721] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.882Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"91c5e53b1102f9b7b112d379","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-91c5e53b1102f9b7b112d379: done: 0.020309 sec","job_status":"done","queue_duration_s":5.892774,"scheduling_latency_s":5.892793,"redis_calls":10,"redis_duration_s":0.002373,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001053,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00132,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012424,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020309,"completed_at":"2026-09-13T10:17:48.882Z","load_balancing_strategy":"primary","db_duration_s":0.002798,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2184gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=456921b37007e803dac208d4de47f6e13ba8b9e4'2185gitlab # [ 427.297652] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.884Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ea0cc3b7ea80e5fff7352d14","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ea0cc3b7ea80e5fff7352d14: start","job_status":"start","queue_duration_s":5.914466,"scheduling_latency_s":5.914484}2186gitlab # [ 427.308973] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.903Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ea0cc3b7ea80e5fff7352d14","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ea0cc3b7ea80e5fff7352d14: done: 0.019575 sec","job_status":"done","queue_duration_s":5.914466,"scheduling_latency_s":5.914484,"redis_calls":10,"redis_duration_s":0.002294,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000949,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001345,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012632,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019575,"completed_at":"2026-09-13T10:17:48.903Z","load_balancing_strategy":"primary","db_duration_s":0.00243,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2187gitlab # [ 427.344822] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.905Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d32214761db8f39bee6d6762","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d32214761db8f39bee6d6762: start","job_status":"start","queue_duration_s":5.935544,"scheduling_latency_s":5.935563}2188gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=456921b37007e803dac208d4de47f6e13ba8b9e4', in 0.07 seconds)2189gitlab # [ 427.356868] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.924Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d32214761db8f39bee6d6762","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d32214761db8f39bee6d6762: done: 0.019619 sec","job_status":"done","queue_duration_s":5.935544,"scheduling_latency_s":5.935563,"redis_calls":10,"redis_duration_s":0.00254,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000952,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001588,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012356,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019619,"completed_at":"2026-09-13T10:17:48.924Z","load_balancing_strategy":"primary","db_duration_s":0.002215,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2190gitlab # [ 427.394611] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.926Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d0b96e94c026e946fb6bbf5d","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d0b96e94c026e946fb6bbf5d: start","job_status":"start","queue_duration_s":5.956584,"scheduling_latency_s":5.956602}2191gitlab # [ 427.406722] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.946Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d0b96e94c026e946fb6bbf5d","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d0b96e94c026e946fb6bbf5d: done: 0.01994 sec","job_status":"done","queue_duration_s":5.956584,"scheduling_latency_s":5.956602,"redis_calls":10,"redis_duration_s":0.002449,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001022,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001427,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012785,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01994,"completed_at":"2026-09-13T10:17:48.946Z","load_balancing_strategy":"primary","db_duration_s":0.002413,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2192gitlab # [ 427.445488] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.947Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f827f4053a6ede234bb915d9","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f827f4053a6ede234bb915d9: start","job_status":"start","queue_duration_s":5.977989,"scheduling_latency_s":5.978008}2193gitlab # [ 427.457856] nixbot[3628]: INFO: ::ffff:127.0.0.1:35124 - "GET /api/repos/gitlab/root/test-flake/builds?commit=456921b37007e803dac208d4de47f6e13ba8b9e4 HTTP/1.1" 200 OK2194gitlab # [ 427.459792] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.968Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f827f4053a6ede234bb915d9","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f827f4053a6ede234bb915d9: done: 0.021019 sec","job_status":"done","queue_duration_s":5.977989,"scheduling_latency_s":5.978008,"redis_calls":10,"redis_duration_s":0.003003,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001083,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00192,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012744,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021019,"completed_at":"2026-09-13T10:17:48.968Z","load_balancing_strategy":"primary","db_duration_s":0.002636,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2195gitlab # [ 427.497692] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.970Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"99e4f4807d2fba9c72002a04","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-99e4f4807d2fba9c72002a04: start","job_status":"start","queue_duration_s":6.000506,"scheduling_latency_s":6.000525}2196gitlab # [ 427.509790] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.990Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"99e4f4807d2fba9c72002a04","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-99e4f4807d2fba9c72002a04: done: 0.019909 sec","job_status":"done","queue_duration_s":6.000506,"scheduling_latency_s":6.000525,"redis_calls":10,"redis_duration_s":0.002504,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001094,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00141,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01265,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019909,"completed_at":"2026-09-13T10:17:48.990Z","load_balancing_strategy":"primary","db_duration_s":0.0026,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2197gitlab # [ 427.547638] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:48.992Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f5dd01530c7713d7a2210089","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f5dd01530c7713d7a2210089: start","job_status":"start","queue_duration_s":6.022405,"scheduling_latency_s":6.022424}2198gitlab # [ 427.559486] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.012Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f5dd01530c7713d7a2210089","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f5dd01530c7713d7a2210089: done: 0.019835 sec","job_status":"done","queue_duration_s":6.022405,"scheduling_latency_s":6.022424,"redis_calls":10,"redis_duration_s":0.0025139999999999997,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001049,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001465,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012424,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019835,"completed_at":"2026-09-13T10:17:49.011Z","load_balancing_strategy":"primary","db_duration_s":0.002658,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2199gitlab # [ 427.597255] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.013Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"563433c5dbb5d77dfba51f62","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-563433c5dbb5d77dfba51f62: start","job_status":"start","queue_duration_s":6.043715,"scheduling_latency_s":6.043732}2200gitlab # [ 427.609216] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.033Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"563433c5dbb5d77dfba51f62","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-563433c5dbb5d77dfba51f62: done: 0.020377 sec","job_status":"done","queue_duration_s":6.043715,"scheduling_latency_s":6.043732,"redis_calls":10,"redis_duration_s":0.002611,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001082,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001529,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012652,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020377,"completed_at":"2026-09-13T10:17:49.033Z","load_balancing_strategy":"primary","db_duration_s":0.003047,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2201gitlab # [ 427.646717] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.035Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d0862ba8647e80fd958f3fd3","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d0862ba8647e80fd958f3fd3: start","job_status":"start","queue_duration_s":6.065534,"scheduling_latency_s":6.065552}2202gitlab # [ 427.660081] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.055Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d0862ba8647e80fd958f3fd3","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d0862ba8647e80fd958f3fd3: done: 0.020059 sec","job_status":"done","queue_duration_s":6.065534,"scheduling_latency_s":6.065552,"redis_calls":10,"redis_duration_s":0.0024349999999999997,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000983,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001452,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012096,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020059,"completed_at":"2026-09-13T10:17:49.055Z","load_balancing_strategy":"primary","db_duration_s":0.002706,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2203gitlab # [ 427.697607] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.056Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e0d855662b03d3ae447e5006","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e0d855662b03d3ae447e5006: start","job_status":"start","queue_duration_s":6.086986,"scheduling_latency_s":6.087003}2204gitlab # [ 427.709584] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.077Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e0d855662b03d3ae447e5006","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e0d855662b03d3ae447e5006: done: 0.020598 sec","job_status":"done","queue_duration_s":6.086986,"scheduling_latency_s":6.087003,"redis_calls":10,"redis_duration_s":0.002445,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001029,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001416,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013343,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020598,"completed_at":"2026-09-13T10:17:49.077Z","load_balancing_strategy":"primary","db_duration_s":0.002624,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2205gitlab # [ 427.747017] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.078Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9283e6c86d3274200457f2bc","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9283e6c86d3274200457f2bc: start","job_status":"start","queue_duration_s":6.109178,"scheduling_latency_s":6.109197}2206gitlab # [ 427.758897] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.104Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9283e6c86d3274200457f2bc","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9283e6c86d3274200457f2bc: done: 0.025853 sec","job_status":"done","queue_duration_s":6.109178,"scheduling_latency_s":6.109197,"redis_calls":10,"redis_duration_s":0.0029839999999999997,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001204,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00178,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.003,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017246,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025853,"completed_at":"2026-09-13T10:17:49.104Z","load_balancing_strategy":"primary","db_duration_s":0.004184,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2207gitlab # [ 427.796726] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.106Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"fb2c748b77d1740a67ae4aeb","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-fb2c748b77d1740a67ae4aeb: start","job_status":"start","queue_duration_s":6.136819,"scheduling_latency_s":6.136837}2208gitlab # [ 427.808498] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.126Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"fb2c748b77d1740a67ae4aeb","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-fb2c748b77d1740a67ae4aeb: done: 0.020121 sec","job_status":"done","queue_duration_s":6.136819,"scheduling_latency_s":6.136837,"redis_calls":10,"redis_duration_s":0.002527,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001056,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001471,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012654,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020121,"completed_at":"2026-09-13T10:17:49.126Z","load_balancing_strategy":"primary","db_duration_s":0.002818,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2209gitlab # [ 427.845912] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.128Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"0474dadfa8f9ad0cfc6e2d4e","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-0474dadfa8f9ad0cfc6e2d4e: start","job_status":"start","queue_duration_s":6.158391,"scheduling_latency_s":6.15841}2210gitlab # [ 427.858751] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.149Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"0474dadfa8f9ad0cfc6e2d4e","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-0474dadfa8f9ad0cfc6e2d4e: done: 0.021738 sec","job_status":"done","queue_duration_s":6.158391,"scheduling_latency_s":6.15841,"redis_calls":10,"redis_duration_s":0.0024920000000000003,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00103,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001462,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0143,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021738,"completed_at":"2026-09-13T10:17:49.149Z","load_balancing_strategy":"primary","db_duration_s":0.002662,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2211gitlab # [ 427.896591] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.151Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4cc8b3ba05d2f284c0bb89df","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4cc8b3ba05d2f284c0bb89df: start","job_status":"start","queue_duration_s":6.181601,"scheduling_latency_s":6.181619}2212gitlab # [ 427.908387] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.172Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4cc8b3ba05d2f284c0bb89df","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4cc8b3ba05d2f284c0bb89df: done: 0.020639 sec","job_status":"done","queue_duration_s":6.181601,"scheduling_latency_s":6.181619,"redis_calls":10,"redis_duration_s":0.0025180000000000003,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001047,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001471,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013091,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020639,"completed_at":"2026-09-13T10:17:49.171Z","load_balancing_strategy":"primary","db_duration_s":0.002883,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2213gitlab # [ 427.946587] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.173Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"0021747be62a714e2b41743f","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-0021747be62a714e2b41743f: start","job_status":"start","queue_duration_s":6.203674,"scheduling_latency_s":6.203691}2214gitlab # [ 427.958809] gitaly[966]: time="2026-09-13T10:17:49.773Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2D4929ZQRKFKW6TJVV0RNR0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-09-13T16:17:49.760" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:17:49.761" grpc.time_ms=11.548 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2215gitlab # [ 427.969442] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.193Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"0021747be62a714e2b41743f","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-0021747be62a714e2b41743f: done: 0.020483 sec","job_status":"done","queue_duration_s":6.203674,"scheduling_latency_s":6.203691,"redis_calls":10,"redis_duration_s":0.0024000000000000002,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001026,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001374,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013274,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020483,"completed_at":"2026-09-13T10:17:49.193Z","load_balancing_strategy":"primary","db_duration_s":0.002694,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2216gitlab # [ 428.007710] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.195Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"95b590986cbfae9ff3ea24e7","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-95b590986cbfae9ff3ea24e7: start","job_status":"start","queue_duration_s":6.225556,"scheduling_latency_s":6.225575}2217gitlab # [ 428.019823] gitaly[966]: time="2026-09-13T10:17:49.811Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=596 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4929ZQRKFKW6TJVV0RNR0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-13T10:18:19.592" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:17:49.793" grpc.time_ms=17.976 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2218gitlab # [ 428.032973] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.219Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"95b590986cbfae9ff3ea24e7","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-95b590986cbfae9ff3ea24e7: done: 0.023788 sec","job_status":"done","queue_duration_s":6.225556,"scheduling_latency_s":6.225575,"redis_calls":10,"redis_duration_s":0.002641,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001101,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00154,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013595,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023788,"completed_at":"2026-09-13T10:17:49.219Z","load_balancing_strategy":"primary","db_duration_s":0.002781,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2219gitlab # [ 428.071252] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.220Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"0e3fcdf9ba9caa230dc3a4ca","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-0e3fcdf9ba9caa230dc3a4ca: start","job_status":"start","queue_duration_s":6.250766,"scheduling_latency_s":6.250784}2220gitlab # [ 428.085088] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.240Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"0e3fcdf9ba9caa230dc3a4ca","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-0e3fcdf9ba9caa230dc3a4ca: done: 0.020115 sec","job_status":"done","queue_duration_s":6.250766,"scheduling_latency_s":6.250784,"redis_calls":10,"redis_duration_s":0.0024920000000000003,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001062,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00143,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013063,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020115,"completed_at":"2026-09-13T10:17:49.240Z","load_balancing_strategy":"primary","db_duration_s":0.002561,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2221gitlab # [ 428.123237] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.242Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5ad2ef0a1b0a3a8cce5e6907","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5ad2ef0a1b0a3a8cce5e6907: start","job_status":"start","queue_duration_s":6.272601,"scheduling_latency_s":6.272618}2222gitlab # [ 428.135476] gitaly[966]: time="2026-09-13T10:17:49.960Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2D4929ZQRKFKW6TJVV0RNR0 diskcache=7fdc8d00-d737-49c3-9c61-c53afaf5cb5d grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-13T16:17:49.916" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:17:49.918" pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2223gitlab # [ 428.145241] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.283Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5ad2ef0a1b0a3a8cce5e6907","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5ad2ef0a1b0a3a8cce5e6907: done: 0.041295 sec","job_status":"done","queue_duration_s":6.272601,"scheduling_latency_s":6.272618,"redis_calls":10,"redis_duration_s":0.003671,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001404,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002267,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.004,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.004,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02976,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.041295,"completed_at":"2026-09-13T10:17:49.283Z","load_balancing_strategy":"primary","db_duration_s":0.005399,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2224gitlab # [ 428.183236] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.286Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"0695b9ff86df8aee809bfc50","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-0695b9ff86df8aee809bfc50: start","job_status":"start","queue_duration_s":6.317022,"scheduling_latency_s":6.317068}2225gitlab # [ 428.195712] gitaly[966]: time="2026-09-13T10:17:49.960Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=28 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1176 command.oublock=0 command.real_time_ms=49 command.spawn_token_fork_ms=4 command.system_time_ms=12 command.user_time_ms=16 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4929ZQRKFKW6TJVV0RNR0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-13T16:17:49.916" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:17:49.918" grpc.time_ms=42.206 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2226gitlab # [ 428.210971] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.325Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"0695b9ff86df8aee809bfc50","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-0695b9ff86df8aee809bfc50: done: 0.038885 sec","job_status":"done","queue_duration_s":6.317022,"scheduling_latency_s":6.317068,"redis_calls":10,"redis_duration_s":0.003469,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001467,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002002,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026444,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.038885,"completed_at":"2026-09-13T10:17:49.325Z","load_balancing_strategy":"primary","db_duration_s":0.003328,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2227gitlab # [ 428.248691] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.327Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"2a770267ff1463cf78341c3f","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-2a770267ff1463cf78341c3f: start","job_status":"start","queue_duration_s":6.358111,"scheduling_latency_s":6.358144}2228gitlab # [ 428.260431] gitaly[966]: time="2026-09-13T10:17:50.097Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=596 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=2 command.system_time_ms=0 command.user_time_ms=12 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4929ZQRKFKW6TJVV0RNR0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-13T10:18:20.081" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:17:50.081" grpc.time_ms=15.287 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2229gitlab # [ 428.273424] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.361Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"2a770267ff1463cf78341c3f","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-2a770267ff1463cf78341c3f: done: 0.03402 sec","job_status":"done","queue_duration_s":6.358111,"scheduling_latency_s":6.358144,"redis_calls":10,"redis_duration_s":0.003299,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001484,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001815,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022613,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.03402,"completed_at":"2026-09-13T10:17:49.361Z","load_balancing_strategy":"primary","db_duration_s":0.003132,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2230gitlab # [ 428.313032] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.364Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ab2f1fcfce83d7f491cc4a84","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ab2f1fcfce83d7f491cc4a84: start","job_status":"start","queue_duration_s":6.394906,"scheduling_latency_s":6.394939}2231gitlab # [ 428.325030] gitaly[966]: time="2026-09-13T10:17:50.174Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4929ZQRKFKW6TJVV0RNR0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:18:20.172" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:17:50.172" grpc.time_ms=1.348 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2232gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=456921b37007e803dac208d4de47f6e13ba8b9e4'2233gitlab # [ 428.337376] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.395Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ab2f1fcfce83d7f491cc4a84","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ab2f1fcfce83d7f491cc4a84: done: 0.031097 sec","job_status":"done","queue_duration_s":6.394906,"scheduling_latency_s":6.394939,"redis_calls":10,"redis_duration_s":0.0028840000000000003,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00115,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001734,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020463,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031097,"completed_at":"2026-09-13T10:17:49.395Z","load_balancing_strategy":"primary","db_duration_s":0.002948,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2234gitlab # [ 428.375864] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.397Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f7a1f207afc7e7b89c1fa153","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f7a1f207afc7e7b89c1fa153: start","job_status":"start","queue_duration_s":6.427819,"scheduling_latency_s":6.427847}2235gitlab # [ 428.387400] gitaly[966]: time="2026-09-13T10:17:50.196Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4929ZQRKFKW6TJVV0RNR0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-13T10:18:00.078" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:17:50.178" grpc.time_ms=17.26 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2236gitlab # [ 428.399646] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.423Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f7a1f207afc7e7b89c1fa153","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f7a1f207afc7e7b89c1fa153: done: 0.025788 sec","job_status":"done","queue_duration_s":6.427819,"scheduling_latency_s":6.427847,"redis_calls":10,"redis_duration_s":0.002791,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001154,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001637,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016807,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025788,"completed_at":"2026-09-13T10:17:49.423Z","load_balancing_strategy":"primary","db_duration_s":0.002677,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2237gitlab # [ 428.436918] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.425Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6b35a31a50ef93b7cef4cc60","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6b35a31a50ef93b7cef4cc60: start","job_status":"start","queue_duration_s":6.455874,"scheduling_latency_s":6.4559}2238gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=456921b37007e803dac208d4de47f6e13ba8b9e4', in 0.10 seconds)2239gitlab # [ 428.449053] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.449Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6b35a31a50ef93b7cef4cc60","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6b35a31a50ef93b7cef4cc60: done: 0.023663 sec","job_status":"done","queue_duration_s":6.455874,"scheduling_latency_s":6.4559,"redis_calls":10,"redis_duration_s":0.0026100000000000003,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001095,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001515,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01552,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023663,"completed_at":"2026-09-13T10:17:49.449Z","load_balancing_strategy":"primary","db_duration_s":0.002609,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2240gitlab # [ 428.487472] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.450Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"07d8ce31ab0444c55ad66425","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-07d8ce31ab0444c55ad66425: start","job_status":"start","queue_duration_s":6.481153,"scheduling_latency_s":6.481176}2241gitlab # [ 428.501166] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.474Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"07d8ce31ab0444c55ad66425","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-07d8ce31ab0444c55ad66425: done: 0.023567 sec","job_status":"done","queue_duration_s":6.481153,"scheduling_latency_s":6.481176,"redis_calls":10,"redis_duration_s":0.003439,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001578,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001861,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014924,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023567,"completed_at":"2026-09-13T10:17:49.474Z","load_balancing_strategy":"primary","db_duration_s":0.002592,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2242gitlab # [ 428.539538] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.476Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"887c016b8a4482d063a9506f","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-887c016b8a4482d063a9506f: start","job_status":"start","queue_duration_s":6.50654,"scheduling_latency_s":6.506562}2243gitlab # [ 428.552051] nixbot[3628]: INFO: ::ffff:127.0.0.1:35138 - "GET /api/repos/gitlab/root/test-flake/builds?commit=456921b37007e803dac208d4de47f6e13ba8b9e4 HTTP/1.1" 200 OK2244gitlab # [ 428.553982] gitaly[966]: time="2026-09-13T10:17:50.383Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D4929ZQRKFKW6TJVV0RNR0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-13T10:18:20.381" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:17:50.382" grpc.time_ms=1.355 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2245gitlab # [ 428.566764] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.497Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"887c016b8a4482d063a9506f","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-887c016b8a4482d063a9506f: done: 0.021301 sec","job_status":"done","queue_duration_s":6.50654,"scheduling_latency_s":6.506562,"redis_calls":10,"redis_duration_s":0.002584,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001065,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001519,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013315,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021301,"completed_at":"2026-09-13T10:17:49.497Z","load_balancing_strategy":"primary","db_duration_s":0.002502,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2246gitlab # [ 428.605004] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.499Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6eb8cd14797aef5a99ec526c","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6eb8cd14797aef5a99ec526c: start","job_status":"start","queue_duration_s":6.529277,"scheduling_latency_s":6.529295}2247gitlab # [ 428.616995] gitaly[966]: time="2026-09-13T10:17:50.392Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=28 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M2D4929ZQRKFKW6TJVV0RNR0 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-13T10:18:20.387" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:17:50.388" grpc.time_ms=4.369 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2248gitlab # [ 428.633109] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.519Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6eb8cd14797aef5a99ec526c","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6eb8cd14797aef5a99ec526c: done: 0.020422 sec","job_status":"done","queue_duration_s":6.529277,"scheduling_latency_s":6.529295,"redis_calls":10,"redis_duration_s":0.002563,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001089,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001474,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013148,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020422,"completed_at":"2026-09-13T10:17:49.519Z","load_balancing_strategy":"primary","db_duration_s":0.002774,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2249gitlab # [ 428.671418] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.520Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"8020839e4edd9f3cb3fbaca7","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-8020839e4edd9f3cb3fbaca7: start","job_status":"start","queue_duration_s":6.551129,"scheduling_latency_s":6.551147}2250gitlab # [ 428.683835] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.541Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"8020839e4edd9f3cb3fbaca7","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-8020839e4edd9f3cb3fbaca7: done: 0.02046 sec","job_status":"done","queue_duration_s":6.551129,"scheduling_latency_s":6.551147,"redis_calls":10,"redis_duration_s":0.0025199999999999997,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001076,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001444,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012854,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02046,"completed_at":"2026-09-13T10:17:49.541Z","load_balancing_strategy":"primary","db_duration_s":0.002454,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2251gitlab # [ 428.723819] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.542Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6221a9439ab48ba602de4e58","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6221a9439ab48ba602de4e58: start","job_status":"start","queue_duration_s":6.573049,"scheduling_latency_s":6.573066}2252gitlab # [ 428.736201] gitaly[966]: time="2026-09-13T10:17:50.567Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2D4929ZQRKFKW6TJVV0RNR0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:18:00.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:17:50.549" grpc.time_ms=17.46 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2253gitlab # [ 428.749261] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.562Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6221a9439ab48ba602de4e58","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6221a9439ab48ba602de4e58: done: 0.019598 sec","job_status":"done","queue_duration_s":6.573049,"scheduling_latency_s":6.573066,"redis_calls":10,"redis_duration_s":0.0024460000000000003,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000988,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001458,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012547,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019598,"completed_at":"2026-09-13T10:17:49.562Z","load_balancing_strategy":"primary","db_duration_s":0.00244,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2254gitlab # [ 428.787727] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.563Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9508e7a788326d63c6532615","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9508e7a788326d63c6532615: start","job_status":"start","queue_duration_s":6.594133,"scheduling_latency_s":6.594152}2255gitlab # [ 428.800099] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.585Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9508e7a788326d63c6532615","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9508e7a788326d63c6532615: done: 0.021723 sec","job_status":"done","queue_duration_s":6.594133,"scheduling_latency_s":6.594152,"redis_calls":10,"redis_duration_s":0.002911,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001028,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001883,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014255,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021723,"completed_at":"2026-09-13T10:17:49.585Z","load_balancing_strategy":"primary","db_duration_s":0.002514,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2256gitlab # [ 428.838052] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.587Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b54f4af9ce1ba59b263280cb","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b54f4af9ce1ba59b263280cb: start","job_status":"start","queue_duration_s":6.617279,"scheduling_latency_s":6.617297}2257gitlab # [ 428.850187] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.609Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b54f4af9ce1ba59b263280cb","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b54f4af9ce1ba59b263280cb: done: 0.022032 sec","job_status":"done","queue_duration_s":6.617279,"scheduling_latency_s":6.617297,"redis_calls":10,"redis_duration_s":0.002918,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001221,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001697,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.003,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014515,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022032,"completed_at":"2026-09-13T10:17:49.609Z","load_balancing_strategy":"primary","db_duration_s":0.002964,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2258gitlab # [ 428.888508] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.611Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b1b667accc017535372af9e6","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b1b667accc017535372af9e6: start","job_status":"start","queue_duration_s":6.641455,"scheduling_latency_s":6.641475}2259gitlab # [ 428.900596] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.636Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b1b667accc017535372af9e6","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b1b667accc017535372af9e6: done: 0.024876 sec","job_status":"done","queue_duration_s":6.641455,"scheduling_latency_s":6.641475,"redis_calls":10,"redis_duration_s":0.002801,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001103,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001698,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016345,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024876,"completed_at":"2026-09-13T10:17:49.636Z","load_balancing_strategy":"primary","db_duration_s":0.002442,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2260gitlab # [ 428.940141] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.637Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"21769c8102a6782bc09914f4","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-21769c8102a6782bc09914f4: start","job_status":"start","queue_duration_s":6.667894,"scheduling_latency_s":6.667911}2261gitlab # [ 428.952235] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.658Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"21769c8102a6782bc09914f4","created_at":"2026-09-13T10:17:42.869Z","trace_propagation_headers":{"sentry-trace":"57be6ff2c2884e89a01a563136dec468-8ba969edc6634d8b","baggage":"sentry-trace_id=57be6ff2c2884e89a01a563136dec468,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"71b9a665c76842bd9a5a9074bcc2a9a9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:42.969Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-21769c8102a6782bc09914f4: done: 0.020793 sec","job_status":"done","queue_duration_s":6.667894,"scheduling_latency_s":6.667911,"redis_calls":10,"redis_duration_s":0.002467,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001031,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001436,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013574,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020793,"completed_at":"2026-09-13T10:17:49.658Z","load_balancing_strategy":"primary","db_duration_s":0.002431,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2262gitlab # [ 428.990405] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.660Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"2359f465cea068c7491854ba","created_at":"2026-09-13T10:17:45.302Z","trace_propagation_headers":{"sentry-trace":"c9b02740ec404b059c9837f1b36e1571-7791d0fdeee74479","baggage":"sentry-trace_id=c9b02740ec404b059c9837f1b36e1571,sentry-release=Unknown"},"correlation_id":"01M2D4929ZQRKFKW6TJVV0RNR0","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:45.313Z","job_size_bytes":46,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-2359f465cea068c7491854ba: start","job_status":"start","queue_duration_s":4.346785,"scheduling_latency_s":4.346807}2263gitlab # [ 429.006157] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.869Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"2359f465cea068c7491854ba","created_at":"2026-09-13T10:17:45.302Z","trace_propagation_headers":{"sentry-trace":"c9b02740ec404b059c9837f1b36e1571-7791d0fdeee74479","baggage":"sentry-trace_id=c9b02740ec404b059c9837f1b36e1571,sentry-release=Unknown"},"correlation_id":"01M2D4929ZQRKFKW6TJVV0RNR0","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:45.313Z","job_size_bytes":46,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-2359f465cea068c7491854ba: done: 0.209656 sec","job_status":"done","queue_duration_s":4.346785,"scheduling_latency_s":4.346807,"gitaly_calls":2,"gitaly_duration_s":0.039318,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.013552,"redis_read_bytes":69,"redis_write_bytes":4888,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.00248,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000975,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2116,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002682,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.006203,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":753,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001212,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.142436,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00063554,"duration_s":0.209656,"completed_at":"2026-09-13T10:17:49.869Z","load_balancing_strategy":"primary","db_duration_s":0.02265,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2264gitlab # [ 429.053705] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:49.872Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"6b9ba9a9ba3f84af63bf62c5","created_at":"2026-09-13T10:17:45.347Z","trace_propagation_headers":{"sentry-trace":"c9b02740ec404b059c9837f1b36e1571-7791d0fdeee74479","baggage":"sentry-trace_id=c9b02740ec404b059c9837f1b36e1571,sentry-release=Unknown"},"correlation_id":"01M2D4929ZQRKFKW6TJVV0RNR0","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:45.359Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-6b9ba9a9ba3f84af63bf62c5: start","job_status":"start","queue_duration_s":4.512968,"scheduling_latency_s":4.513002}2265gitlab # [ 429.070125] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:50.028Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"6b9ba9a9ba3f84af63bf62c5","created_at":"2026-09-13T10:17:45.347Z","trace_propagation_headers":{"sentry-trace":"c9b02740ec404b059c9837f1b36e1571-7791d0fdeee74479","baggage":"sentry-trace_id=c9b02740ec404b059c9837f1b36e1571,sentry-release=Unknown"},"correlation_id":"01M2D4929ZQRKFKW6TJVV0RNR0","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:45.359Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-6b9ba9a9ba3f84af63bf62c5: done: 0.156301 sec","job_status":"done","queue_duration_s":4.512968,"scheduling_latency_s":4.513002,"gitaly_calls":1,"gitaly_duration_s":0.047938,"redis_calls":15,"redis_duration_s":0.0057870000000000005,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000683,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002167,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001173,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001764,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.019,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.036,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.074845,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.11504332,"duration_s":0.156301,"completed_at":"2026-09-13T10:17:50.028Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.027491,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2266gitlab # [ 429.116106] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:50.031Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"a25f161a918814dca4175972","created_at":"2026-09-13T10:17:45.547Z","trace_propagation_headers":{"sentry-trace":"c9b02740ec404b059c9837f1b36e1571-7791d0fdeee74479","baggage":"sentry-trace_id=c9b02740ec404b059c9837f1b36e1571,sentry-release=Unknown"},"correlation_id":"01M2D4929ZQRKFKW6TJVV0RNR0","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:45.550Z","job_size_bytes":23,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-a25f161a918814dca4175972: start","job_status":"start","queue_duration_s":4.481255,"scheduling_latency_s":4.481287}2267gitlab # [ 429.131417] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:50.118Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"a25f161a918814dca4175972","created_at":"2026-09-13T10:17:45.547Z","trace_propagation_headers":{"sentry-trace":"c9b02740ec404b059c9837f1b36e1571-7791d0fdeee74479","baggage":"sentry-trace_id=c9b02740ec404b059c9837f1b36e1571,sentry-release=Unknown"},"correlation_id":"01M2D4929ZQRKFKW6TJVV0RNR0","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:45.550Z","job_size_bytes":23,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-a25f161a918814dca4175972: done: 0.08733 sec","job_status":"done","queue_duration_s":4.481255,"scheduling_latency_s":4.481287,"gitaly_calls":1,"gitaly_duration_s":0.019905,"redis_calls":16,"redis_duration_s":0.006654,"redis_read_bytes":52,"redis_write_bytes":2049,"redis_queues_calls":1,"redis_queues_duration_s":0.000594,"redis_queues_read_bytes":1,"redis_queues_write_bytes":980,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001876,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.003036,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001148,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.054947,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.08733,"completed_at":"2026-09-13T10:17:50.118Z","load_balancing_strategy":"primary","db_duration_s":0.00501,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2268gitlab # [ 429.177782] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:50.122Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"25522fa1183930fb45f53d9a","created_at":"2026-09-13T10:17:46.502Z","trace_propagation_headers":{"sentry-trace":"c9b02740ec404b059c9837f1b36e1571-7791d0fdeee74479","baggage":"sentry-trace_id=c9b02740ec404b059c9837f1b36e1571,sentry-release=Unknown"},"correlation_id":"01M2D4929ZQRKFKW6TJVV0RNR0","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:62f8f1bfbf5b80ba8521eb0ac63a965627ede6e4a9b0f08ecfb50cd760fe0d7f","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:46.511Z","job_size_bytes":467,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-25522fa1183930fb45f53d9a: start","job_status":"start","queue_duration_s":3.610525,"scheduling_latency_s":3.610567}2269gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=456921b37007e803dac208d4de47f6e13ba8b9e4'2270gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=456921b37007e803dac208d4de47f6e13ba8b9e4', in 0.08 seconds)2271gitlab # [ 429.534427] gitaly[966]: time="2026-09-13T10:17:51.442Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=881 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2D4929ZQRKFKW6TJVV0RNR0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-13T10:18:21.420" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:17:51.421" grpc.time_ms=20.589 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2272gitlab # [ 429.579358] gitaly[966]: time="2026-09-13T10:17:51.471Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D4929ZQRKFKW6TJVV0RNR0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-13T10:18:21.447" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:17:51.447" grpc.time_ms=23.305 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2273gitlab # [ 429.610764] nixbot[3628]: INFO: ::ffff:127.0.0.1:35140 - "GET /api/repos/gitlab/root/test-flake/builds?commit=456921b37007e803dac208d4de47f6e13ba8b9e4 HTTP/1.1" 200 OK2274gitlab # [ 429.622687] sidekiq-cluster[3504]: {"severity":"WARN","time":"2026-09-13T10:17:51.530Z","class":"CreatePipelineWorker","correlation_id":"01M2D4929ZQRKFKW6TJVV0RNR0","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"a97336d9b6bb4bb908a8849d973a90d364eab2ac","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"a97336d9b6bb4bb908a8849d973a90d364eab2ac","ref":"refs/heads/master"},"retry":0}2275gitlab # [ 429.642071] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:51.549Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"25522fa1183930fb45f53d9a","created_at":"2026-09-13T10:17:46.502Z","trace_propagation_headers":{"sentry-trace":"c9b02740ec404b059c9837f1b36e1571-7791d0fdeee74479","baggage":"sentry-trace_id=c9b02740ec404b059c9837f1b36e1571,sentry-release=Unknown"},"correlation_id":"01M2D4929ZQRKFKW6TJVV0RNR0","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:62f8f1bfbf5b80ba8521eb0ac63a965627ede6e4a9b0f08ecfb50cd760fe0d7f","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:46.511Z","job_size_bytes":467,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-25522fa1183930fb45f53d9a: done: 1.427907 sec","job_status":"done","queue_duration_s":3.610525,"scheduling_latency_s":3.610567,"gitaly_calls":7,"gitaly_duration_s":0.111232,"redis_calls":51,"redis_duration_s":0.016551,"redis_read_bytes":169,"redis_write_bytes":4168,"redis_cache_calls":4,"redis_cache_duration_s":0.00159,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.006653,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002334,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.004249,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001725,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.051,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.044,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":1.230003,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00079748,"duration_s":1.427907,"completed_at":"2026-09-13T10:17:51.549Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.089272,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2276gitlab # [ 429.718522] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:51.553Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"68a759f7-2e4f-4cba-9555-861b41f40e46\"}"],"class":"WebHookWorker","jid":"dbfcec41a3b57d77391a0fa8","created_at":"2026-09-13T10:17:46.701Z","trace_propagation_headers":{"sentry-trace":"c9b02740ec404b059c9837f1b36e1571-7791d0fdeee74479","baggage":"sentry-trace_id=c9b02740ec404b059c9837f1b36e1571,sentry-release=Unknown"},"correlation_id":"01M2D4929ZQRKFKW6TJVV0RNR0","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:315240299d2be38db06fc6497f65a5712aa16811761a6713f9ca78c956029352","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:46.707Z","job_size_bytes":1751,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-dbfcec41a3b57d77391a0fa8: start","job_status":"start","queue_duration_s":4.846187,"scheduling_latency_s":4.846233}2277gitlab # [ 429.936743] nixbot[3628]: INFO: ::ffff:127.0.0.1:35154 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted2278gitlab # [ 429.974318] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:51.882Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"68a759f7-2e4f-4cba-9555-861b41f40e46\"}"],"class":"WebHookWorker","jid":"dbfcec41a3b57d77391a0fa8","created_at":"2026-09-13T10:17:46.701Z","trace_propagation_headers":{"sentry-trace":"c9b02740ec404b059c9837f1b36e1571-7791d0fdeee74479","baggage":"sentry-trace_id=c9b02740ec404b059c9837f1b36e1571,sentry-release=Unknown"},"correlation_id":"01M2D4929ZQRKFKW6TJVV0RNR0","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:315240299d2be38db06fc6497f65a5712aa16811761a6713f9ca78c956029352","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:46.707Z","job_size_bytes":1751,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-dbfcec41a3b57d77391a0fa8: done: 0.32857 sec","job_status":"done","queue_duration_s":4.846187,"scheduling_latency_s":4.846233,"redis_calls":19,"redis_duration_s":0.007348,"redis_read_bytes":32,"redis_write_bytes":5368,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001874,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000778,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3730,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002954,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001742,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.09195361999996976,"cpu_s":0.195544,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00063212,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.32857,"completed_at":"2026-09-13T10:17:51.882Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.024304,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2279gitlab # [ 430.072427] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:51.885Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"4975cc8d844d946d8ed800ec","created_at":"2026-09-13T10:17:47.180Z","trace_propagation_headers":{"sentry-trace":"b857f0cecad048e38efbe21aef5b640d-3f6931d77b9741c6","baggage":"sentry-trace_id=b857f0cecad048e38efbe21aef5b640d,sentry-release=Unknown"},"correlation_id":"01M2D498P87CGPP5YH2KQ33ZAW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:47.186Z","job_size_bytes":23,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-4975cc8d844d946d8ed800ec: start","job_status":"start","queue_duration_s":4.69921,"scheduling_latency_s":4.699249}2280gitlab # [ 430.112432] gitaly[966]: time="2026-09-13T10:17:52.021Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=4 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D4BY0S8ASTH1H01QYT50SV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:17:52.010" grpc.time_ms=11.037 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc2281gitlab # [ 430.138407] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:17:52 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1242282gitlab # [ 430.141987] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:52.039Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"4975cc8d844d946d8ed800ec","created_at":"2026-09-13T10:17:47.180Z","trace_propagation_headers":{"sentry-trace":"b857f0cecad048e38efbe21aef5b640d-3f6931d77b9741c6","baggage":"sentry-trace_id=b857f0cecad048e38efbe21aef5b640d,sentry-release=Unknown"},"correlation_id":"01M2D498P87CGPP5YH2KQ33ZAW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:47.186Z","job_size_bytes":23,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-4975cc8d844d946d8ed800ec: done: 0.153523 sec","job_status":"done","queue_duration_s":4.69921,"scheduling_latency_s":4.699249,"redis_calls":10,"redis_duration_s":0.004224,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000787,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002673,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000764,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.008,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.008,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.114915,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00059504,"duration_s":0.153523,"completed_at":"2026-09-13T10:17:52.039Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.027667,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2283gitlab # [ 430.204541] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:52.042Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"6baa0bb5fd2817b9d3009b25","created_at":"2026-09-13T10:17:47.350Z","trace_propagation_headers":{"sentry-trace":"b857f0cecad048e38efbe21aef5b640d-3f6931d77b9741c6","baggage":"sentry-trace_id=b857f0cecad048e38efbe21aef5b640d,sentry-release=Unknown"},"correlation_id":"01M2D498P87CGPP5YH2KQ33ZAW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:47.362Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-6baa0bb5fd2817b9d3009b25: start","job_status":"start","queue_duration_s":4.67917,"scheduling_latency_s":4.679208}2284gitlab # [ 430.221947] gitaly[966]: time="2026-09-13T10:17:52.110Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2D4BY4RX9F7KWRRJ77E0GTV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:17:52.094" pid=966 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2285gitlab # [ 430.232005] gitlab-workhorse[885]: time="2026-09-13T10:17:52Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672286gitlab # [ 430.233824] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:17:52 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 862287gitlab # [ 430.235672] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:52.122Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"6baa0bb5fd2817b9d3009b25","created_at":"2026-09-13T10:17:47.350Z","trace_propagation_headers":{"sentry-trace":"b857f0cecad048e38efbe21aef5b640d-3f6931d77b9741c6","baggage":"sentry-trace_id=b857f0cecad048e38efbe21aef5b640d,sentry-release=Unknown"},"correlation_id":"01M2D498P87CGPP5YH2KQ33ZAW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:47.362Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-6baa0bb5fd2817b9d3009b25: done: 0.080902 sec","job_status":"done","queue_duration_s":4.67917,"scheduling_latency_s":4.679208,"redis_calls":10,"redis_duration_s":0.007724999999999999,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000645,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002815,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004265,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.042602,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00181768,"duration_s":0.080902,"completed_at":"2026-09-13T10:17:52.122Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011222,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2288gitlab # [ 430.279665] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:52.125Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"9e0912c31d19b66ef98b77d7","created_at":"2026-09-13T10:17:47.391Z","trace_propagation_headers":{"sentry-trace":"b857f0cecad048e38efbe21aef5b640d-3f6931d77b9741c6","baggage":"sentry-trace_id=b857f0cecad048e38efbe21aef5b640d,sentry-release=Unknown"},"correlation_id":"01M2D498P87CGPP5YH2KQ33ZAW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:47.405Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-9e0912c31d19b66ef98b77d7: start","job_status":"start","queue_duration_s":4.719966,"scheduling_latency_s":4.719987}2289gitlab # [ 430.295173] gitaly[966]: time="2026-09-13T10:17:52.110Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=5 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4BY4RX9F7KWRRJ77E0GTV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:17:52.094" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc2290gitlab # [ 430.308470] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:52.174Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"9e0912c31d19b66ef98b77d7","created_at":"2026-09-13T10:17:47.391Z","trace_propagation_headers":{"sentry-trace":"b857f0cecad048e38efbe21aef5b640d-3f6931d77b9741c6","baggage":"sentry-trace_id=b857f0cecad048e38efbe21aef5b640d,sentry-release=Unknown"},"correlation_id":"01M2D498P87CGPP5YH2KQ33ZAW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:47.405Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-9e0912c31d19b66ef98b77d7: done: 0.04946 sec","job_status":"done","queue_duration_s":4.719966,"scheduling_latency_s":4.719987,"redis_calls":10,"redis_duration_s":0.003522,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001044,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001874,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000604,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029735,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00049354,"duration_s":0.04946,"completed_at":"2026-09-13T10:17:52.174Z","load_balancing_strategy":"primary","db_duration_s":0.011001,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2291gitlab # [ 430.352201] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:52.177Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"5d9dfa7a3788c5cf1620cf05","created_at":"2026-09-13T10:17:47.433Z","trace_propagation_headers":{"sentry-trace":"b857f0cecad048e38efbe21aef5b640d-3f6931d77b9741c6","baggage":"sentry-trace_id=b857f0cecad048e38efbe21aef5b640d,sentry-release=Unknown"},"correlation_id":"01M2D498P87CGPP5YH2KQ33ZAW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:47.443Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-5d9dfa7a3788c5cf1620cf05: start","job_status":"start","queue_duration_s":4.734012,"scheduling_latency_s":4.734034}2292gitlab # [ 430.368825] gitaly[966]: time="2026-09-13T10:17:52.110Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=5 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4BY4RX9F7KWRRJ77E0GTV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:17:52.094" grpc.time_ms=15.637 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc2293gitlab # [ 430.382738] gitlab-workhorse[885]: time="2026-09-13T10:17:52Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3402294gitlab # [ 430.384585] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:17:52 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 340 "" "git/2.55.0" 1082295gitlab # [ 430.386464] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:52.230Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"5d9dfa7a3788c5cf1620cf05","created_at":"2026-09-13T10:17:47.433Z","trace_propagation_headers":{"sentry-trace":"b857f0cecad048e38efbe21aef5b640d-3f6931d77b9741c6","baggage":"sentry-trace_id=b857f0cecad048e38efbe21aef5b640d,sentry-release=Unknown"},"correlation_id":"01M2D498P87CGPP5YH2KQ33ZAW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:47.443Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-5d9dfa7a3788c5cf1620cf05: done: 0.05297 sec","job_status":"done","queue_duration_s":4.734012,"scheduling_latency_s":4.734034,"redis_calls":13,"redis_duration_s":0.004902,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000627,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002763,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00076,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000752,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029226,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00168646,"duration_s":0.05297,"completed_at":"2026-09-13T10:17:52.230Z","load_balancing_strategy":"primary","db_duration_s":0.014818,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2296gitlab # [ 430.430727] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:52.231Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"a1f9914de181411a525f2409","created_at":"2026-09-13T10:17:47.444Z","trace_propagation_headers":{"sentry-trace":"b857f0cecad048e38efbe21aef5b640d-3f6931d77b9741c6","baggage":"sentry-trace_id=b857f0cecad048e38efbe21aef5b640d,sentry-release=Unknown"},"correlation_id":"01M2D498P87CGPP5YH2KQ33ZAW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:47.453Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-a1f9914de181411a525f2409: start","job_status":"start","queue_duration_s":4.778187,"scheduling_latency_s":4.778207}2297gitlab # [ 430.447000] gitaly[966]: time="2026-09-13T10:17:52.217Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2D4BY7HR8TT00GBRGMVE9BZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:17:52.192" pid=966 remote_ip="2001:db8:1::1" request_sha=160a9d7e0b3494ed352f1ba2c31b1a84fbf8467a response_bytes=340 span.kind=server system=grpc2298gitlab # [ 430.457095] nixbot[3628]: {"ts": "2026-09-13T10:17:52.290Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"}2299gitlab # [ 430.459498] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:52.277Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"a1f9914de181411a525f2409","created_at":"2026-09-13T10:17:47.444Z","trace_propagation_headers":{"sentry-trace":"b857f0cecad048e38efbe21aef5b640d-3f6931d77b9741c6","baggage":"sentry-trace_id=b857f0cecad048e38efbe21aef5b640d,sentry-release=Unknown"},"correlation_id":"01M2D498P87CGPP5YH2KQ33ZAW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:47.453Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-a1f9914de181411a525f2409: done: 0.045499 sec","job_status":"done","queue_duration_s":4.778187,"scheduling_latency_s":4.778207,"redis_calls":10,"redis_duration_s":0.001379,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000297,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000802,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00028,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024704,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00023472,"duration_s":0.045499,"completed_at":"2026-09-13T10:17:52.277Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014667,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2300gitlab # [ 430.503038] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:52.278Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"ac52a277a4a072105787408f","created_at":"2026-09-13T10:17:47.454Z","trace_propagation_headers":{"sentry-trace":"b857f0cecad048e38efbe21aef5b640d-3f6931d77b9741c6","baggage":"sentry-trace_id=b857f0cecad048e38efbe21aef5b640d,sentry-release=Unknown"},"correlation_id":"01M2D498P87CGPP5YH2KQ33ZAW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:47.464Z","job_size_bytes":19,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-ac52a277a4a072105787408f: start","job_status":"start","queue_duration_s":4.814154,"scheduling_latency_s":4.814175}2301gitlab # [ 430.519091] gitaly[966]: time="2026-09-13T10:17:52.217Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1131 command.oublock=0 command.real_time_ms=20 command.spawn_token_fork_ms=0 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4BY7HR8TT00GBRGMVE9BZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:17:52.192" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc2302gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=456921b37007e803dac208d4de47f6e13ba8b9e4'2303gitlab # [ 430.532683] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:52.320Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"ac52a277a4a072105787408f","created_at":"2026-09-13T10:17:47.454Z","trace_propagation_headers":{"sentry-trace":"b857f0cecad048e38efbe21aef5b640d-3f6931d77b9741c6","baggage":"sentry-trace_id=b857f0cecad048e38efbe21aef5b640d,sentry-release=Unknown"},"correlation_id":"01M2D498P87CGPP5YH2KQ33ZAW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:47.464Z","job_size_bytes":19,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-ac52a277a4a072105787408f: done: 0.041893 sec","job_status":"done","queue_duration_s":4.814154,"scheduling_latency_s":4.814175,"redis_calls":11,"redis_duration_s":0.006252000000000001,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000803,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.005015,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000434,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02186,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00024184,"duration_s":0.041893,"completed_at":"2026-09-13T10:17:52.320Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007577,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2304gitlab # [ 430.575754] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:52.327Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"8eb3552db5a1275f452f91ff","created_at":"2026-09-13T10:17:47.470Z","trace_propagation_headers":{"sentry-trace":"b857f0cecad048e38efbe21aef5b640d-3f6931d77b9741c6","baggage":"sentry-trace_id=b857f0cecad048e38efbe21aef5b640d,sentry-release=Unknown"},"correlation_id":"01M2D498P87CGPP5YH2KQ33ZAW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:47.478Z","job_size_bytes":66,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-8eb3552db5a1275f452f91ff: start","job_status":"start","queue_duration_s":4.849028,"scheduling_latency_s":4.849074}2305gitlab # [ 430.596395] gitaly[966]: time="2026-09-13T10:17:52.220Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1131 command.oublock=0 command.real_time_ms=20 command.spawn_token_fork_ms=0 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4BY7HR8TT00GBRGMVE9BZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:17:52.192" grpc.time_ms=27.366 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc2306gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=456921b37007e803dac208d4de47f6e13ba8b9e4', in 0.09 seconds)2307gitlab # [ 430.611085] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:52.390Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"8eb3552db5a1275f452f91ff","created_at":"2026-09-13T10:17:47.470Z","trace_propagation_headers":{"sentry-trace":"b857f0cecad048e38efbe21aef5b640d-3f6931d77b9741c6","baggage":"sentry-trace_id=b857f0cecad048e38efbe21aef5b640d,sentry-release=Unknown"},"correlation_id":"01M2D498P87CGPP5YH2KQ33ZAW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:47.478Z","job_size_bytes":66,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-8eb3552db5a1275f452f91ff: done: 0.063273 sec","job_status":"done","queue_duration_s":4.849028,"scheduling_latency_s":4.849074,"redis_calls":10,"redis_duration_s":0.013791,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003596,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.008791,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001404,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028422,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00409098,"duration_s":0.063273,"completed_at":"2026-09-13T10:17:52.390Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012023,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2308gitlab # [ 430.655866] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:52.395Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"645038821070d8d00d635943","created_at":"2026-09-13T10:17:47.479Z","trace_propagation_headers":{"sentry-trace":"b857f0cecad048e38efbe21aef5b640d-3f6931d77b9741c6","baggage":"sentry-trace_id=b857f0cecad048e38efbe21aef5b640d,sentry-release=Unknown"},"correlation_id":"01M2D498P87CGPP5YH2KQ33ZAW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:47.487Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-645038821070d8d00d635943: start","job_status":"start","queue_duration_s":4.908083,"scheduling_latency_s":4.908105}2309gitlab # [ 430.671675] gitaly[966]: time="2026-09-13T10:17:52.403Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4BYDZ3XJ64WWR7RP4SE15 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:18:22.401" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:17:52.402" grpc.time_ms=1.166 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2310gitlab # [ 430.684384] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:17:52 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/a97336d9b6bb4bb908a8849d973a90d364eab2ac HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 2412311gitlab # [ 430.686837] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:52.457Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"645038821070d8d00d635943","created_at":"2026-09-13T10:17:47.479Z","trace_propagation_headers":{"sentry-trace":"b857f0cecad048e38efbe21aef5b640d-3f6931d77b9741c6","baggage":"sentry-trace_id=b857f0cecad048e38efbe21aef5b640d,sentry-release=Unknown"},"correlation_id":"01M2D498P87CGPP5YH2KQ33ZAW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:47.487Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-645038821070d8d00d635943: done: 0.06182 sec","job_status":"done","queue_duration_s":4.908083,"scheduling_latency_s":4.908105,"redis_calls":10,"redis_duration_s":0.009998,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000491,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.008998,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000509,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033998,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0005002,"duration_s":0.06182,"completed_at":"2026-09-13T10:17:52.457Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012167,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2312gitlab # [ 430.729744] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:52.461Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"8aed6ba9a128ebe0fb80df73","created_at":"2026-09-13T10:17:47.497Z","trace_propagation_headers":{"sentry-trace":"b857f0cecad048e38efbe21aef5b640d-3f6931d77b9741c6","baggage":"sentry-trace_id=b857f0cecad048e38efbe21aef5b640d,sentry-release=Unknown"},"correlation_id":"01M2D498P87CGPP5YH2KQ33ZAW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:47.502Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-8aed6ba9a128ebe0fb80df73: start","job_status":"start","queue_duration_s":4.958838,"scheduling_latency_s":4.958862}2313gitlab # [ 430.746225] nixbot[3628]: INFO: ::ffff:127.0.0.1:35170 - "GET /api/repos/gitlab/root/test-flake/builds?commit=456921b37007e803dac208d4de47f6e13ba8b9e4 HTTP/1.1" 200 OK2314gitlab # [ 430.748140] nixbot[3628]: {"ts": "2026-09-13T10:17:52.563Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/a97336d9b6bb4bb908a8849d973a90d364eab2ac \"HTTP/1.1 201 Created\""}2315gitlab # [ 430.750820] gitaly[966]: time="2026-09-13T10:17:52.498Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=6 command.system_time_ms=0 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01M2D4BYDZ3XJ64WWR7RP4SE15 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:18:02.076" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:17:52.486" grpc.time_ms=10.944 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2316gitlab # [ 430.765650] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:52.519Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"8aed6ba9a128ebe0fb80df73","created_at":"2026-09-13T10:17:47.497Z","trace_propagation_headers":{"sentry-trace":"b857f0cecad048e38efbe21aef5b640d-3f6931d77b9741c6","baggage":"sentry-trace_id=b857f0cecad048e38efbe21aef5b640d,sentry-release=Unknown"},"correlation_id":"01M2D498P87CGPP5YH2KQ33ZAW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:47.502Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-8aed6ba9a128ebe0fb80df73: done: 0.057436 sec","job_status":"done","queue_duration_s":4.958838,"scheduling_latency_s":4.958862,"redis_calls":10,"redis_duration_s":0.0051199999999999996,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00045,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002881,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001789,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035886,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00052438,"duration_s":0.057436,"completed_at":"2026-09-13T10:17:52.519Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010278,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2317gitlab # [ 430.809410] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:52.522Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"fb6f2ed9ca2bddd5517f52ea","created_at":"2026-09-13T10:17:47.503Z","trace_propagation_headers":{"sentry-trace":"b857f0cecad048e38efbe21aef5b640d-3f6931d77b9741c6","baggage":"sentry-trace_id=b857f0cecad048e38efbe21aef5b640d,sentry-release=Unknown"},"correlation_id":"01M2D498P87CGPP5YH2KQ33ZAW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:47.508Z","job_size_bytes":6,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-fb6f2ed9ca2bddd5517f52ea: start","job_status":"start","queue_duration_s":5.013748,"scheduling_latency_s":5.013781}2318gitlab # [ 430.825740] gitaly[966]: time="2026-09-13T10:17:52.672Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4BYNT4KNYZD4MJE09A1CM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:18:22.071" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:17:52.672" grpc.time_ms=0.656 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2319gitlab # [ 430.838064] gitaly[966]: time="2026-09-13T10:17:52.729Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2D4BJPHAMGG58F3SJZ3VXBD diskcache=40735904-2e23-4d80-99a7-7120340ba42c grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-13T16:17:52.717" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:17:52.717" pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2320gitlab # [ 430.847691] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:52.563Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"fb6f2ed9ca2bddd5517f52ea","created_at":"2026-09-13T10:17:47.503Z","trace_propagation_headers":{"sentry-trace":"b857f0cecad048e38efbe21aef5b640d-3f6931d77b9741c6","baggage":"sentry-trace_id=b857f0cecad048e38efbe21aef5b640d,sentry-release=Unknown"},"correlation_id":"01M2D498P87CGPP5YH2KQ33ZAW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:47.508Z","job_size_bytes":6,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-fb6f2ed9ca2bddd5517f52ea: done: 0.040476 sec","job_status":"done","queue_duration_s":5.013748,"scheduling_latency_s":5.013781,"redis_calls":10,"redis_duration_s":0.003396,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000733,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002252,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000411,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023137,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00039934,"duration_s":0.040476,"completed_at":"2026-09-13T10:17:52.562Z","load_balancing_strategy":"primary","db_duration_s":0.002575,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2321gitlab # [ 430.890872] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:52.564Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"dd7c78935d210b70200ab58a","created_at":"2026-09-13T10:17:47.509Z","trace_propagation_headers":{"sentry-trace":"b857f0cecad048e38efbe21aef5b640d-3f6931d77b9741c6","baggage":"sentry-trace_id=b857f0cecad048e38efbe21aef5b640d,sentry-release=Unknown"},"correlation_id":"01M2D498P87CGPP5YH2KQ33ZAW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:47.515Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-dd7c78935d210b70200ab58a: start","job_status":"start","queue_duration_s":5.049154,"scheduling_latency_s":5.049181}2322gitlab # [ 430.906062] gitaly[966]: time="2026-09-13T10:17:52.730Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=289 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4BJPHAMGG58F3SJZ3VXBD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-13T16:17:52.717" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:17:52.717" grpc.time_ms=12.415 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2323gitlab # [ 430.919237] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:52.607Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"dd7c78935d210b70200ab58a","created_at":"2026-09-13T10:17:47.509Z","trace_propagation_headers":{"sentry-trace":"b857f0cecad048e38efbe21aef5b640d-3f6931d77b9741c6","baggage":"sentry-trace_id=b857f0cecad048e38efbe21aef5b640d,sentry-release=Unknown"},"correlation_id":"01M2D498P87CGPP5YH2KQ33ZAW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:47.515Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-dd7c78935d210b70200ab58a: done: 0.042653 sec","job_status":"done","queue_duration_s":5.049154,"scheduling_latency_s":5.049181,"redis_calls":10,"redis_duration_s":0.001836,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000312,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001022,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000502,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02828,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00028076,"duration_s":0.042653,"completed_at":"2026-09-13T10:17:52.607Z","load_balancing_strategy":"primary","db_duration_s":0.007383,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2324gitlab # [ 430.961352] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:52.608Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"c9ec7fe1332757209a8d218b","created_at":"2026-09-13T10:17:47.523Z","trace_propagation_headers":{"sentry-trace":"b857f0cecad048e38efbe21aef5b640d-3f6931d77b9741c6","baggage":"sentry-trace_id=b857f0cecad048e38efbe21aef5b640d,sentry-release=Unknown"},"correlation_id":"01M2D498P87CGPP5YH2KQ33ZAW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:47.529Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-c9ec7fe1332757209a8d218b: start","job_status":"start","queue_duration_s":5.079481,"scheduling_latency_s":5.079498}2325gitlab # [ 430.978599] nixbot[3628]: {"ts": "2026-09-13T10:17:52.822Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/a97336d9b6bb4bb908a8849d973a90d364eab2ac \"HTTP/1.1 201 Created\""}2326gitlab # [ 430.981351] gitaly[966]: time="2026-09-13T10:17:52.764Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D4BYNT4KNYZD4MJE09A1CM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:18:02.175" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:17:52.755" grpc.time_ms=8.815 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2327gitlab # [ 430.994502] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:17:52 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/a97336d9b6bb4bb908a8849d973a90d364eab2ac HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 2502328gitlab # [ 430.997057] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:52.682Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"c9ec7fe1332757209a8d218b","created_at":"2026-09-13T10:17:47.523Z","trace_propagation_headers":{"sentry-trace":"b857f0cecad048e38efbe21aef5b640d-3f6931d77b9741c6","baggage":"sentry-trace_id=b857f0cecad048e38efbe21aef5b640d,sentry-release=Unknown"},"correlation_id":"01M2D498P87CGPP5YH2KQ33ZAW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:47.529Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-c9ec7fe1332757209a8d218b: done: 0.073449 sec","job_status":"done","queue_duration_s":5.079481,"scheduling_latency_s":5.079498,"redis_calls":14,"redis_duration_s":0.007599,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000418,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005195,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001986,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04953,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00106362,"duration_s":0.073449,"completed_at":"2026-09-13T10:17:52.682Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011384,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2329gitlab # [ 431.040057] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:52.685Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"f0e9f45a599adb8a80fdd796","created_at":"2026-09-13T10:17:47.798Z","trace_propagation_headers":{"sentry-trace":"bb684fb7a0c4498faf7f0ac698ac4cca-a5f8c8240c084193","baggage":"sentry-trace_id=bb684fb7a0c4498faf7f0ac698ac4cca,sentry-release=Unknown"},"correlation_id":"01M2D4BJPHAMGG58F3SJZ3VXBD","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"6b9ba9a9ba3f84af63bf62c5","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:47.803Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-f0e9f45a599adb8a80fdd796: start","job_status":"start","queue_duration_s":4.881784,"scheduling_latency_s":4.881809}2330gitlab # [ 431.056657] gitaly[966]: time="2026-09-13T10:17:52.833Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4BJPHAMGG58F3SJZ3VXBD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:18:22.831" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:17:52.832" grpc.time_ms=0.979 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2331gitlab # [ 431.068840] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:52.772Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"f0e9f45a599adb8a80fdd796","created_at":"2026-09-13T10:17:47.798Z","trace_propagation_headers":{"sentry-trace":"bb684fb7a0c4498faf7f0ac698ac4cca-a5f8c8240c084193","baggage":"sentry-trace_id=bb684fb7a0c4498faf7f0ac698ac4cca,sentry-release=Unknown"},"correlation_id":"01M2D4BJPHAMGG58F3SJZ3VXBD","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"6b9ba9a9ba3f84af63bf62c5","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:47.803Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-f0e9f45a599adb8a80fdd796: done: 0.086197 sec","job_status":"done","queue_duration_s":4.881784,"scheduling_latency_s":4.881809,"gitaly_calls":1,"gitaly_duration_s":0.01581,"redis_calls":12,"redis_duration_s":0.00383,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001544,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000897,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001389,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.015,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.015,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04987,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.04923246,"duration_s":0.086197,"completed_at":"2026-09-13T10:17:52.771Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01537,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2332gitlab # [ 431.113293] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:52.777Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"a240e518d881f57b0c06646e","created_at":"2026-09-13T10:17:48.025Z","trace_propagation_headers":{"sentry-trace":"bb684fb7a0c4498faf7f0ac698ac4cca-a5f8c8240c084193","baggage":"sentry-trace_id=bb684fb7a0c4498faf7f0ac698ac4cca,sentry-release=Unknown"},"correlation_id":"01M2D4BJPHAMGG58F3SJZ3VXBD","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8d20a15ca6e7f48817a293d6b7efa1bff2f0aa16184c152cf4de4f536b011584","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:48.030Z","job_size_bytes":467,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-a240e518d881f57b0c06646e: start","job_status":"start","queue_duration_s":4.746621,"scheduling_latency_s":4.746656}2333gitlab # [ 431.130279] gitaly[966]: time="2026-09-13T10:17:52.848Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4BJPHAMGG58F3SJZ3VXBD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-13T10:18:02.007" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:17:52.838" grpc.time_ms=9.689 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2334gitlab # [ 431.143179] gitaly[966]: time="2026-09-13T10:17:52.920Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D4BJPHAMGG58F3SJZ3VXBD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-13T10:18:22.016" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:17:52.918" grpc.time_ms=1.592 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2335gitlab # [ 431.155910] gitaly[966]: time="2026-09-13T10:17:52.933Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=203 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D4BJPHAMGG58F3SJZ3VXBD error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-13T10:18:22.026" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:17:52.926" grpc.time_ms=6.019 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2336gitlab # [ 431.172876] gitaly[966]: time="2026-09-13T10:17:52.941Z" level=info msg="finished unary call with code OK" catfile.duration_ms=5 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=5 catfile.request_object_count=1 catfile.request_object_ms=0 command.spawn_token_fork_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4BYXVB35PC3DG9K738JS6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:18:22.030" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:17:52.930" grpc.time_ms=10.279 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2337gitlab # [ 431.185948] nixbot[3628]: {"ts": "2026-09-13T10:17:53.079Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/a97336d9b6bb4bb908a8849d973a90d364eab2ac \"HTTP/1.1 201 Created\""}2338gitlab # [ 431.189058] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:17:53 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/a97336d9b6bb4bb908a8849d973a90d364eab2ac HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 2502339gitlab # [ 431.191535] gitaly[966]: time="2026-09-13T10:17:53.005Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=3 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D4BYXVB35PC3DG9K738JS6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:18:02.169" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:17:52.989" grpc.time_ms=15.799 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2340gitlab # [ 431.205186] gitaly[966]: time="2026-09-13T10:17:53.008Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=1 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D4BJPHAMGG58F3SJZ3VXBD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:18:02.161" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:17:52.991" grpc.time_ms=16.573 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2341gitlab # [ 431.315798] gitaly[966]: time="2026-09-13T10:17:53.224Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=1 command.system_time_ms=16 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D4BZ7G9KN1W2H8X9M5ZRF3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:17:53.204" grpc.time_ms=19.879 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc2342gitlab # [ 431.343801] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:17:53 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 882343gitlab # [ 431.446180] gitaly[966]: time="2026-09-13T10:17:53.354Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2D4BZAB5KK7026839Q00FYW grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:17:53.333" pid=966 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2344gitlab # [ 431.474997] gitaly[966]: time="2026-09-13T10:17:53.355Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4BZAB5KK7026839Q00FYW grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:17:53.333" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc2345gitlab # [ 431.501257] gitlab-workhorse[885]: time="2026-09-13T10:17:53Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672346gitlab # [ 431.504281] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:17:53 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1292347gitlab # [ 431.507228] gitaly[966]: time="2026-09-13T10:17:53.355Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4BZAB5KK7026839Q00FYW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:17:53.333" grpc.time_ms=22.334 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc2348gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=456921b37007e803dac208d4de47f6e13ba8b9e4'2349gitlab # [ 431.658376] gitaly[966]: time="2026-09-13T10:17:53.566Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=881 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=1 command.system_time_ms=12 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2D4BJPHAMGG58F3SJZ3VXBD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-13T10:18:23.544" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:17:53.544" grpc.time_ms=21.662 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2350gitlab # [ 431.683737] gitaly[966]: time="2026-09-13T10:17:53.592Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01M2D4BJPHAMGG58F3SJZ3VXBD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-13T10:18:23.571" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:17:53.571" grpc.time_ms=20.669 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2351gitlab # [ 431.709540] sidekiq-cluster[3504]: {"severity":"WARN","time":"2026-09-13T10:17:53.616Z","class":"CreatePipelineWorker","correlation_id":"01M2D4BJPHAMGG58F3SJZ3VXBD","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"a97336d9b6bb4bb908a8849d973a90d364eab2ac","after":"456921b37007e803dac208d4de47f6e13ba8b9e4","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"456921b37007e803dac208d4de47f6e13ba8b9e4","ref":"refs/heads/master"},"retry":0}2352gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=456921b37007e803dac208d4de47f6e13ba8b9e4', in 0.08 seconds)2353gitlab # [ 431.721406] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:53.624Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"a240e518d881f57b0c06646e","created_at":"2026-09-13T10:17:48.025Z","trace_propagation_headers":{"sentry-trace":"bb684fb7a0c4498faf7f0ac698ac4cca-a5f8c8240c084193","baggage":"sentry-trace_id=bb684fb7a0c4498faf7f0ac698ac4cca,sentry-release=Unknown"},"correlation_id":"01M2D4BJPHAMGG58F3SJZ3VXBD","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8d20a15ca6e7f48817a293d6b7efa1bff2f0aa16184c152cf4de4f536b011584","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:48.030Z","job_size_bytes":467,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-a240e518d881f57b0c06646e: done: 0.847395 sec","job_status":"done","queue_duration_s":4.746621,"scheduling_latency_s":4.746656,"gitaly_calls":7,"gitaly_duration_s":0.109283,"redis_calls":33,"redis_duration_s":0.017843,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.005124,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002353,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.008066,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.0023,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.038,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.03,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.66381,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.847395,"completed_at":"2026-09-13T10:17:53.624Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.036971,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2354gitlab # [ 431.770968] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:53.627Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"80e19678-948d-44ef-9f12-13940e6de550\"}"],"class":"WebHookWorker","jid":"53a452224c6d83253da21cd6","created_at":"2026-09-13T10:17:48.066Z","trace_propagation_headers":{"sentry-trace":"bb684fb7a0c4498faf7f0ac698ac4cca-a5f8c8240c084193","baggage":"sentry-trace_id=bb684fb7a0c4498faf7f0ac698ac4cca,sentry-release=Unknown"},"correlation_id":"01M2D4BJPHAMGG58F3SJZ3VXBD","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b9de0f5a1a0ce50afde3b14859e7b38d3a1276709dd720707d5abd873e9b30cf","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:48.071Z","job_size_bytes":1749,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-53a452224c6d83253da21cd6: start","job_status":"start","queue_duration_s":5.556162,"scheduling_latency_s":5.556198}2355gitlab # [ 431.788812] nixbot[3628]: INFO: ::ffff:127.0.0.1:56856 - "GET /api/repos/gitlab/root/test-flake/builds?commit=456921b37007e803dac208d4de47f6e13ba8b9e4 HTTP/1.1" 200 OK2356gitlab # [ 431.790698] nixbot[3628]: INFO: ::ffff:127.0.0.1:56862 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted2357gitlab # [ 431.791931] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:53.691Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"80e19678-948d-44ef-9f12-13940e6de550\"}"],"class":"WebHookWorker","jid":"53a452224c6d83253da21cd6","created_at":"2026-09-13T10:17:48.066Z","trace_propagation_headers":{"sentry-trace":"bb684fb7a0c4498faf7f0ac698ac4cca-a5f8c8240c084193","baggage":"sentry-trace_id=bb684fb7a0c4498faf7f0ac698ac4cca,sentry-release=Unknown"},"correlation_id":"01M2D4BJPHAMGG58F3SJZ3VXBD","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b9de0f5a1a0ce50afde3b14859e7b38d3a1276709dd720707d5abd873e9b30cf","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:48.071Z","job_size_bytes":1749,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-53a452224c6d83253da21cd6: done: 0.064243 sec","job_status":"done","queue_duration_s":5.556162,"scheduling_latency_s":5.556198,"redis_calls":14,"redis_duration_s":0.006565,"redis_read_bytes":25,"redis_write_bytes":4684,"redis_queues_calls":2,"redis_queues_duration_s":0.000644,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3728,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002225,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003696,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.008978319999982887,"cpu_s":0.036883,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.064243,"completed_at":"2026-09-13T10:17:53.691Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003408,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2358gitlab # [ 431.839799] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:53.695Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"9ccacae85458e63ea29d15b8","created_at":"2026-09-13T10:17:49.833Z","trace_propagation_headers":{"sentry-trace":"c9b02740ec404b059c9837f1b36e1571-58d341be4cc74248","baggage":"sentry-trace_id=c9b02740ec404b059c9837f1b36e1571,sentry-release=Unknown"},"correlation_id":"01M2D4929ZQRKFKW6TJVV0RNR0","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:49.846Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-9ccacae85458e63ea29d15b8: start","job_status":"start","queue_duration_s":3.849639,"scheduling_latency_s":3.849686}2359gitlab # [ 431.901285] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:53.808Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"9ccacae85458e63ea29d15b8","created_at":"2026-09-13T10:17:49.833Z","trace_propagation_headers":{"sentry-trace":"c9b02740ec404b059c9837f1b36e1571-58d341be4cc74248","baggage":"sentry-trace_id=c9b02740ec404b059c9837f1b36e1571,sentry-release=Unknown"},"correlation_id":"01M2D4929ZQRKFKW6TJVV0RNR0","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:49.846Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-9ccacae85458e63ea29d15b8: done: 0.11323 sec","job_status":"done","queue_duration_s":3.849639,"scheduling_latency_s":3.849686,"redis_calls":19,"redis_duration_s":0.00905,"redis_read_bytes":21,"redis_write_bytes":4291,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00152,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.00081,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2262,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.005551,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001169,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.007,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.082186,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01764842,"duration_s":0.11323,"completed_at":"2026-09-13T10:17:53.808Z","load_balancing_strategy":"primary","db_duration_s":0.01348,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2360gitlab # [ 431.962356] gitaly[966]: time="2026-09-13T10:17:53.818Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2D4BZSJH1SH3T4PJ5XTYHXH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:17:53.805" grpc.time_ms=12.357 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc2361gitlab # [ 431.975752] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:17:53 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1032362gitlab # [ 431.977917] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:53.812Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","dbfcec41a3b57d77391a0fa8"],"class":"WebHooks::LogExecutionWorker","jid":"6bf4c73bc327a82a0069a9bf","created_at":"2026-09-13T10:17:51.851Z","trace_propagation_headers":{"sentry-trace":"c9b02740ec404b059c9837f1b36e1571-dcbd6b6127944301","baggage":"sentry-trace_id=c9b02740ec404b059c9837f1b36e1571,sentry-release=Unknown"},"correlation_id":"01M2D4929ZQRKFKW6TJVV0RNR0","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1af87bfb8c0c11373c980f6d2e8fd24492e0fa8ab5a4c073c023b7774035c39a","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:51.864Z","job_size_bytes":2593,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-6bf4c73bc327a82a0069a9bf: start","job_status":"start","queue_duration_s":1.948393,"scheduling_latency_s":1.948432}2363gitlab # [ 432.037009] gitaly[966]: time="2026-09-13T10:17:53.945Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2D4BZWWPR6WCGNP9Y78Q4SC grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:17:53.930" pid=966 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2364gitlab # [ 432.057370] gitaly[966]: time="2026-09-13T10:17:53.946Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4BZWWPR6WCGNP9Y78Q4SC grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:17:53.930" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc2365gitlab # [ 432.077967] gitlab-workhorse[885]: time="2026-09-13T10:17:53Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672366gitlab # [ 432.080478] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:17:53 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1262367gitlab # [ 432.082929] gitaly[966]: time="2026-09-13T10:17:53.946Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4BZWWPR6WCGNP9Y78Q4SC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:17:53.930" grpc.time_ms=15.703 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc2368gitlab # [ 432.119559] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:54.028Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","dbfcec41a3b57d77391a0fa8"],"class":"WebHooks::LogExecutionWorker","jid":"6bf4c73bc327a82a0069a9bf","created_at":"2026-09-13T10:17:51.851Z","trace_propagation_headers":{"sentry-trace":"c9b02740ec404b059c9837f1b36e1571-dcbd6b6127944301","baggage":"sentry-trace_id=c9b02740ec404b059c9837f1b36e1571,sentry-release=Unknown"},"correlation_id":"01M2D4929ZQRKFKW6TJVV0RNR0","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1af87bfb8c0c11373c980f6d2e8fd24492e0fa8ab5a4c073c023b7774035c39a","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:51.864Z","job_size_bytes":2593,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-6bf4c73bc327a82a0069a9bf: done: 0.21573 sec","job_status":"done","queue_duration_s":1.948393,"scheduling_latency_s":1.948432,"redis_calls":10,"redis_duration_s":0.004111,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000641,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002945,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000525,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.008,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.008,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.187445,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002979,"duration_s":0.21573,"completed_at":"2026-09-13T10:17:54.028Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01889,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2369gitlab # [ 432.183044] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:54.030Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"2f4a994938051790cf8c48bb","created_at":"2026-09-13T10:17:52.541Z","trace_propagation_headers":{"sentry-trace":"b8edc2c57bb042659310469133d96551-d572f5ab918a453e","baggage":"sentry-trace_id=b8edc2c57bb042659310469133d96551,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D4BYDZ3XJ64WWR7RP4SE15","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:52.545Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-2f4a994938051790cf8c48bb: start","job_status":"start","queue_duration_s":1.484547,"scheduling_latency_s":1.484565}2370gitlab # [ 432.205730] gitaly[966]: time="2026-09-13T10:17:54.114Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D4C031P941828S46D5RRWT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:17:54.106" grpc.time_ms=8.566 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc2371gitlab # [ 432.229272] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:17:54 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 972372gitlab # [ 432.236560] gitaly[966]: time="2026-09-13T10:17:54.145Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D4BYDZ3XJ64WWR7RP4SE15 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:17:54.144" grpc.time_ms=0.733 pid=966 span.kind=server system=grpc2373gitlab # [ 432.255028] gitaly[966]: time="2026-09-13T10:17:54.163Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D4BYDZ3XJ64WWR7RP4SE15 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:17:54.163" grpc.time_ms=0.515 pid=966 span.kind=server system=grpc2374gitlab # [ 432.279413] gitaly[966]: time="2026-09-13T10:17:54.188Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D4BYDZ3XJ64WWR7RP4SE15 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:17:54.188" grpc.time_ms=0.402 pid=966 span.kind=server system=grpc2375gitlab # [ 432.295156] gitaly[966]: time="2026-09-13T10:17:54.199Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2D4BYDZ3XJ64WWR7RP4SE15 diskcache=564c5057-3b08-45b4-9135-1831bbfff0f1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T10:18:24.120" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:17:54.120" pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2376gitlab # [ 432.309255] gitaly[966]: time="2026-09-13T10:17:54.199Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=72 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=6911 command.oublock=0 command.real_time_ms=74 command.spawn_token_fork_ms=0 command.system_time_ms=32 command.user_time_ms=40 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4BYDZ3XJ64WWR7RP4SE15 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T10:18:24.120" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:17:54.120" grpc.time_ms=79.072 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2377gitlab # [ 432.326496] gitaly[966]: time="2026-09-13T10:17:54.231Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2D4C06594HAAANEAYJZVDTJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:17:54.214" pid=966 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc2378gitlab # [ 432.339679] gitlab-workhorse[885]: time="2026-09-13T10:17:54Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2162379gitlab # [ 432.341989] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:17:54 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 1142380gitlab # [ 432.344201] gitaly[966]: time="2026-09-13T10:17:54.231Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1103 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4C06594HAAANEAYJZVDTJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:17:54.214" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc2381gitlab # [ 432.360040] gitaly[966]: time="2026-09-13T10:17:54.231Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1103 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4C06594HAAANEAYJZVDTJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:17:54.214" grpc.time_ms=17.261 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc2382gitlab # [ 432.375833] gitaly[966]: time="2026-09-13T10:17:54.268Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D4BYDZ3XJ64WWR7RP4SE15 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-13T10:18:24.267" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:17:54.267" grpc.time_ms=0.87 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2383gitlab # [ 432.394871] gitaly[966]: time="2026-09-13T10:17:54.303Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4BYDZ3XJ64WWR7RP4SE15 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:18:24.302" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:17:54.303" grpc.time_ms=0.724 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2384gitlab # [ 432.407102] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:54.315Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"2f4a994938051790cf8c48bb","created_at":"2026-09-13T10:17:52.541Z","trace_propagation_headers":{"sentry-trace":"b8edc2c57bb042659310469133d96551-d572f5ab918a453e","baggage":"sentry-trace_id=b8edc2c57bb042659310469133d96551,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D4BYDZ3XJ64WWR7RP4SE15","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:52.545Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":3504,"job_status":"rescheduled","message":"PipelineProcessWorker JID-2f4a994938051790cf8c48bb: rescheduled"}2385gitlab # [ 432.423322] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:54.328Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"2f4a994938051790cf8c48bb","created_at":"2026-09-13T10:17:52.541Z","trace_propagation_headers":{"sentry-trace":"b8edc2c57bb042659310469133d96551-d572f5ab918a453e","baggage":"sentry-trace_id=b8edc2c57bb042659310469133d96551,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D4BYDZ3XJ64WWR7RP4SE15","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:52.545Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-2f4a994938051790cf8c48bb: done: 0.298673 sec","job_status":"done","queue_duration_s":1.484547,"scheduling_latency_s":1.484565,"gitaly_calls":3,"gitaly_duration_s":0.08731,"redis_calls":66,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.019438999999999998,"redis_read_bytes":148,"redis_write_bytes":18013,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.00069,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001331,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.004043,"redis_queues_read_bytes":26,"redis_queues_write_bytes":12312,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.008502,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000355,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000454,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.003374,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.04,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.01,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.01,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.023,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.157726,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.24456442,"duration_s":0.298673,"completed_at":"2026-09-13T10:17:54.328Z","load_balancing_strategy":"primary","db_duration_s":0.041651,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2386gitlab # [ 432.473729] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:54.333Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","53a452224c6d83253da21cd6"],"class":"WebHooks::LogExecutionWorker","jid":"195e95d9edeb490815157d15","created_at":"2026-09-13T10:17:53.672Z","trace_propagation_headers":{"sentry-trace":"bb684fb7a0c4498faf7f0ac698ac4cca-1c9ea14f0ecc44c5","baggage":"sentry-trace_id=bb684fb7a0c4498faf7f0ac698ac4cca,sentry-release=Unknown"},"correlation_id":"01M2D4BJPHAMGG58F3SJZ3VXBD","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:209582de9bb3f4b2709a49583387f8d4187a0e32600983b0a9b08a3445a09348","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:53.675Z","job_size_bytes":2591,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-195e95d9edeb490815157d15: start","job_status":"start","queue_duration_s":0.65757,"scheduling_latency_s":0.657602}2387gitlab # [ 432.490235] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:54.371Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","53a452224c6d83253da21cd6"],"class":"WebHooks::LogExecutionWorker","jid":"195e95d9edeb490815157d15","created_at":"2026-09-13T10:17:53.672Z","trace_propagation_headers":{"sentry-trace":"bb684fb7a0c4498faf7f0ac698ac4cca-1c9ea14f0ecc44c5","baggage":"sentry-trace_id=bb684fb7a0c4498faf7f0ac698ac4cca,sentry-release=Unknown"},"correlation_id":"01M2D4BJPHAMGG58F3SJZ3VXBD","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:209582de9bb3f4b2709a49583387f8d4187a0e32600983b0a9b08a3445a09348","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:53.675Z","job_size_bytes":2591,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-195e95d9edeb490815157d15: done: 0.038499 sec","job_status":"done","queue_duration_s":0.65757,"scheduling_latency_s":0.657602,"redis_calls":7,"redis_duration_s":0.0022059999999999996,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00113,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001076,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025657,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.038499,"completed_at":"2026-09-13T10:17:54.371Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006907,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2388gitlab # [ 432.532487] gitaly[966]: time="2026-09-13T10:17:54.345Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4C0AVJF9ZAFSEVE773FYX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:18:24.342" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:17:54.342" grpc.time_ms=2.196 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2389gitlab # [ 432.545094] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:54.373Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"2eb019ae87705e4341b84042","created_at":"2026-09-13T10:17:53.779Z","trace_propagation_headers":{"sentry-trace":"c9b02740ec404b059c9837f1b36e1571-3ea6ffe6e5404edf","baggage":"sentry-trace_id=c9b02740ec404b059c9837f1b36e1571,sentry-release=Unknown"},"correlation_id":"01M2D4929ZQRKFKW6TJVV0RNR0","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:53.791Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-2eb019ae87705e4341b84042: start","job_status":"start","queue_duration_s":0.582222,"scheduling_latency_s":0.582246}2390gitlab # [ 432.561594] gitaly[966]: time="2026-09-13T10:17:54.367Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=591 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2D4C0AVJF9ZAFSEVE773FYX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-13T10:18:24.356" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:17:54.356" grpc.time_ms=10.599 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2391gitlab # [ 432.575284] gitaly[966]: time="2026-09-13T10:17:54.384Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D4C0AVJF9ZAFSEVE773FYX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:18:04.134" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:17:54.374" grpc.time_ms=10.176 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2392gitlab # [ 432.588497] gitaly[966]: time="2026-09-13T10:17:54.418Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2D4C0AVJF9ZAFSEVE773FYX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-13T10:18:04.177" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:17:54.417" grpc.time_ms=0.328 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2393gitlab # [ 432.599098] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:54.485Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"2eb019ae87705e4341b84042","created_at":"2026-09-13T10:17:53.779Z","trace_propagation_headers":{"sentry-trace":"c9b02740ec404b059c9837f1b36e1571-3ea6ffe6e5404edf","baggage":"sentry-trace_id=c9b02740ec404b059c9837f1b36e1571,sentry-release=Unknown"},"correlation_id":"01M2D4929ZQRKFKW6TJVV0RNR0","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:53.791Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-2eb019ae87705e4341b84042: done: 0.111532 sec","job_status":"done","queue_duration_s":0.582222,"scheduling_latency_s":0.582246,"redis_calls":11,"redis_duration_s":0.00242,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000795,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001235,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00039,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.078956,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025846,"duration_s":0.111532,"completed_at":"2026-09-13T10:17:54.485Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.025861,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2394gitlab # [ 432.645664] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:54.486Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"41aad7de55d8cd7c3ab1bb65","created_at":"2026-09-13T10:17:54.105Z","trace_propagation_headers":{"sentry-trace":"b8edc2c57bb042659310469133d96551-7a9c465ad14f4fcb","baggage":"sentry-trace_id=b8edc2c57bb042659310469133d96551,sentry-release=Unknown"},"correlation_id":"01M2D4BYDZ3XJ64WWR7RP4SE15","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:54.110Z","job_size_bytes":23,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-41aad7de55d8cd7c3ab1bb65: start","job_status":"start","queue_duration_s":0.376175,"scheduling_latency_s":0.376195}2395gitlab # [ 432.691580] gitaly[966]: time="2026-09-13T10:17:54.433Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=4 command.system_time_ms=0 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4C0AVJF9ZAFSEVE773FYX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-13T10:18:04.183" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:17:54.424" grpc.time_ms=8.825 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2396gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=456921b37007e803dac208d4de47f6e13ba8b9e4'2397gitlab # [ 432.719017] gitaly[966]: time="2026-09-13T10:17:54.472Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D4C0AVJF9ZAFSEVE773FYX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4154 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:17:54.471" grpc.time_ms=0.589 pid=966 span.kind=server system=grpc2398gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=456921b37007e803dac208d4de47f6e13ba8b9e4', in 0.06 seconds)2399(finished: subtest: push delivers the webhook, in 91.33 seconds)2400subtest: eval and build post commit statuses2401gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/456921b37007e803dac208d4de47f6e13ba8b9e4/statuses2402gitlab # [ 432.738774] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:54.532Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"41aad7de55d8cd7c3ab1bb65","created_at":"2026-09-13T10:17:54.105Z","trace_propagation_headers":{"sentry-trace":"b8edc2c57bb042659310469133d96551-7a9c465ad14f4fcb","baggage":"sentry-trace_id=b8edc2c57bb042659310469133d96551,sentry-release=Unknown"},"correlation_id":"01M2D4BYDZ3XJ64WWR7RP4SE15","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:54.110Z","job_size_bytes":23,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-41aad7de55d8cd7c3ab1bb65: done: 0.045688 sec","job_status":"done","queue_duration_s":0.376175,"scheduling_latency_s":0.376195,"redis_calls":7,"redis_duration_s":0.00102,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000726,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000294,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032244,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.045688,"completed_at":"2026-09-13T10:17:54.532Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008101,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2403gitlab # [ 432.796565] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:54.533Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"c8f6ed693ca8707e1b3f5c7e","created_at":"2026-09-13T10:17:54.204Z","trace_propagation_headers":{"sentry-trace":"b8edc2c57bb042659310469133d96551-7a9c465ad14f4fcb","baggage":"sentry-trace_id=b8edc2c57bb042659310469133d96551,sentry-release=Unknown"},"correlation_id":"01M2D4BYDZ3XJ64WWR7RP4SE15","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:54.207Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-c8f6ed693ca8707e1b3f5c7e: start","job_status":"start","queue_duration_s":0.326318,"scheduling_latency_s":0.32634}2404gitlab # [ 432.818003] gitaly[966]: time="2026-09-13T10:17:54.514Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D4C0AVJF9ZAFSEVE773FYX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4156 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:17:54.513" grpc.time_ms=0.831 pid=966 span.kind=server system=grpc2405gitlab # [ 432.828900] gitaly[966]: time="2026-09-13T10:17:54.522Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2D4C0AVJF9ZAFSEVE773FYX diskcache=ab79bb1a-0071-4c3e-be32-5f866247d07b grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-13T10:18:04.195" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:17:54.435" pid=966 ref=refs/keep-around/456921b37007e803dac208d4de47f6e13ba8b9e4 remote_ip="2001:db8:1::1" revision=456921b37007e803dac208d4de47f6e13ba8b9e4 span.kind=server system=grpc username=root2406gitlab # [ 432.841829] nixbot[3628]: {"ts": "2026-09-13T10:17:54.620Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/456921b37007e803dac208d4de47f6e13ba8b9e4 \"HTTP/1.1 201 Created\""}2407gitlab # [ 432.844739] nixbot[3628]: {"ts": "2026-09-13T10:17:54.627Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-456921b37007-5"}2408gitlab # [ 432.847075] nixbot[3628]: INFO: ::ffff:127.0.0.1:56872 - "GET /api/repos/gitlab/root/test-flake/builds?commit=456921b37007e803dac208d4de47f6e13ba8b9e4 HTTP/1.1" 200 OK2409gitlab # [ 432.849141] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:17:54 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/456921b37007e803dac208d4de47f6e13ba8b9e4 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 3502410gitlab # [ 432.851597] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:54.575Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"c8f6ed693ca8707e1b3f5c7e","created_at":"2026-09-13T10:17:54.204Z","trace_propagation_headers":{"sentry-trace":"b8edc2c57bb042659310469133d96551-7a9c465ad14f4fcb","baggage":"sentry-trace_id=b8edc2c57bb042659310469133d96551,sentry-release=Unknown"},"correlation_id":"01M2D4BYDZ3XJ64WWR7RP4SE15","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:54.207Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-c8f6ed693ca8707e1b3f5c7e: done: 0.041781 sec","job_status":"done","queue_duration_s":0.326318,"scheduling_latency_s":0.32634,"redis_calls":7,"redis_duration_s":0.004008,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003429,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000579,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021709,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.041781,"completed_at":"2026-09-13T10:17:54.575Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012272,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2411gitlab # [ 432.894402] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:54.577Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"441b23d175b41e82a91dd23c","created_at":"2026-09-13T10:17:54.215Z","trace_propagation_headers":{"sentry-trace":"b8edc2c57bb042659310469133d96551-7a9c465ad14f4fcb","baggage":"sentry-trace_id=b8edc2c57bb042659310469133d96551,sentry-release=Unknown"},"correlation_id":"01M2D4BYDZ3XJ64WWR7RP4SE15","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:54.219Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-441b23d175b41e82a91dd23c: start","job_status":"start","queue_duration_s":0.358199,"scheduling_latency_s":0.358221}2412gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/456921b37007e803dac208d4de47f6e13ba8b9e4/statuses, in 0.12 seconds)2413gitlab # [ 432.909985] gitaly[966]: time="2026-09-13T10:17:54.523Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=81 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5337 command.oublock=8 command.real_time_ms=81 command.spawn_token_fork_ms=4 command.system_time_ms=57 command.user_time_ms=24 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4C0AVJF9ZAFSEVE773FYX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-13T10:18:04.195" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:17:54.435" grpc.time_ms=87.496 pid=966 ref=refs/keep-around/456921b37007e803dac208d4de47f6e13ba8b9e4 remote_ip="2001:db8:1::1" revision=456921b37007e803dac208d4de47f6e13ba8b9e4 span.kind=server system=grpc username=root2414gitlab # [ 432.924419] nixbot[3628]: {"ts": "2026-09-13T10:17:54.755Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-456921b37007-5", "argv0": "bwrap"}2415gitlab # [ 432.926669] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:17:54 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/456921b37007e803dac208d4de47f6e13ba8b9e4/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 852416gitlab # [ 432.929243] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:54.610Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"441b23d175b41e82a91dd23c","created_at":"2026-09-13T10:17:54.215Z","trace_propagation_headers":{"sentry-trace":"b8edc2c57bb042659310469133d96551-7a9c465ad14f4fcb","baggage":"sentry-trace_id=b8edc2c57bb042659310469133d96551,sentry-release=Unknown"},"correlation_id":"01M2D4BYDZ3XJ64WWR7RP4SE15","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:54.219Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-441b23d175b41e82a91dd23c: done: 0.033183 sec","job_status":"done","queue_duration_s":0.358199,"scheduling_latency_s":0.358221,"redis_calls":7,"redis_duration_s":0.001124,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00077,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000354,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023855,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033183,"completed_at":"2026-09-13T10:17:54.610Z","load_balancing_strategy":"primary","db_duration_s":0.005422,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2417gitlab # [ 432.971078] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:54.611Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"ebd87c8baf8656d4932f5412","created_at":"2026-09-13T10:17:54.228Z","trace_propagation_headers":{"sentry-trace":"b8edc2c57bb042659310469133d96551-7a9c465ad14f4fcb","baggage":"sentry-trace_id=b8edc2c57bb042659310469133d96551,sentry-release=Unknown"},"correlation_id":"01M2D4BYDZ3XJ64WWR7RP4SE15","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:54.232Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-ebd87c8baf8656d4932f5412: start","job_status":"start","queue_duration_s":0.379017,"scheduling_latency_s":0.379039}2418gitlab # [ 432.986665] gitaly[966]: time="2026-09-13T10:17:54.527Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4C0AVJF9ZAFSEVE773FYX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:18:24.526" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:17:54.527" grpc.time_ms=0.385 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2419gitlab # [ 432.999291] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:54.654Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"ebd87c8baf8656d4932f5412","created_at":"2026-09-13T10:17:54.228Z","trace_propagation_headers":{"sentry-trace":"b8edc2c57bb042659310469133d96551-7a9c465ad14f4fcb","baggage":"sentry-trace_id=b8edc2c57bb042659310469133d96551,sentry-release=Unknown"},"correlation_id":"01M2D4BYDZ3XJ64WWR7RP4SE15","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:54.232Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-ebd87c8baf8656d4932f5412: done: 0.042464 sec","job_status":"done","queue_duration_s":0.379017,"scheduling_latency_s":0.379039,"redis_calls":10,"redis_duration_s":0.002005,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001013,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000638,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000354,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032355,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.042464,"completed_at":"2026-09-13T10:17:54.654Z","load_balancing_strategy":"primary","db_duration_s":0.005647,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2420gitlab # [ 433.043276] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:54.655Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"b30a0e4e4b45fbcb8794edae","created_at":"2026-09-13T10:17:54.236Z","trace_propagation_headers":{"sentry-trace":"b8edc2c57bb042659310469133d96551-7a9c465ad14f4fcb","baggage":"sentry-trace_id=b8edc2c57bb042659310469133d96551,sentry-release=Unknown"},"correlation_id":"01M2D4BYDZ3XJ64WWR7RP4SE15","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:54.239Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-b30a0e4e4b45fbcb8794edae: start","job_status":"start","queue_duration_s":0.416537,"scheduling_latency_s":0.416568}2421gitlab # [ 433.060240] gitaly[966]: time="2026-09-13T10:17:54.782Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4C0S6S383JD3E9NERJ488 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:18:24.781" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:17:54.781" grpc.time_ms=0.541 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2422gitlab # [ 433.072799] nix-daemon[3797]: accepted connection from pid 4838, user nixbot2423gitlab # [ 433.073781] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:54.692Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"b30a0e4e4b45fbcb8794edae","created_at":"2026-09-13T10:17:54.236Z","trace_propagation_headers":{"sentry-trace":"b8edc2c57bb042659310469133d96551-7a9c465ad14f4fcb","baggage":"sentry-trace_id=b8edc2c57bb042659310469133d96551,sentry-release=Unknown"},"correlation_id":"01M2D4BYDZ3XJ64WWR7RP4SE15","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:54.239Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-b30a0e4e4b45fbcb8794edae: done: 0.036093 sec","job_status":"done","queue_duration_s":0.416537,"scheduling_latency_s":0.416568,"redis_calls":7,"redis_duration_s":0.0015459999999999998,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001138,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000408,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025248,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036093,"completed_at":"2026-09-13T10:17:54.691Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005379,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2424gitlab # [ 433.116180] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:54.694Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"02b1f372bfd1e5f809d8d9e2","created_at":"2026-09-13T10:17:54.240Z","trace_propagation_headers":{"sentry-trace":"b8edc2c57bb042659310469133d96551-7a9c465ad14f4fcb","baggage":"sentry-trace_id=b8edc2c57bb042659310469133d96551,sentry-release=Unknown"},"correlation_id":"01M2D4BYDZ3XJ64WWR7RP4SE15","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:54.242Z","job_size_bytes":19,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-02b1f372bfd1e5f809d8d9e2: start","job_status":"start","queue_duration_s":0.451574,"scheduling_latency_s":0.451603}2425gitlab # [ 433.131999] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:54.731Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"02b1f372bfd1e5f809d8d9e2","created_at":"2026-09-13T10:17:54.240Z","trace_propagation_headers":{"sentry-trace":"b8edc2c57bb042659310469133d96551-7a9c465ad14f4fcb","baggage":"sentry-trace_id=b8edc2c57bb042659310469133d96551,sentry-release=Unknown"},"correlation_id":"01M2D4BYDZ3XJ64WWR7RP4SE15","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:54.242Z","job_size_bytes":19,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-02b1f372bfd1e5f809d8d9e2: done: 0.03784 sec","job_status":"done","queue_duration_s":0.451574,"scheduling_latency_s":0.451603,"redis_calls":8,"redis_duration_s":0.0015819999999999999,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001218,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000364,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02343,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.03784,"completed_at":"2026-09-13T10:17:54.731Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005213,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2426gitlab # [ 433.174429] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:54.733Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"b624529ccf2ca9fa91f7d6f8","created_at":"2026-09-13T10:17:54.244Z","trace_propagation_headers":{"sentry-trace":"b8edc2c57bb042659310469133d96551-7a9c465ad14f4fcb","baggage":"sentry-trace_id=b8edc2c57bb042659310469133d96551,sentry-release=Unknown"},"correlation_id":"01M2D4BYDZ3XJ64WWR7RP4SE15","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:54.247Z","job_size_bytes":66,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-b624529ccf2ca9fa91f7d6f8: start","job_status":"start","queue_duration_s":0.486418,"scheduling_latency_s":0.486447}2427gitlab # [ 433.191485] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:54.760Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"b624529ccf2ca9fa91f7d6f8","created_at":"2026-09-13T10:17:54.244Z","trace_propagation_headers":{"sentry-trace":"b8edc2c57bb042659310469133d96551-7a9c465ad14f4fcb","baggage":"sentry-trace_id=b8edc2c57bb042659310469133d96551,sentry-release=Unknown"},"correlation_id":"01M2D4BYDZ3XJ64WWR7RP4SE15","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:54.247Z","job_size_bytes":66,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-b624529ccf2ca9fa91f7d6f8: done: 0.026577 sec","job_status":"done","queue_duration_s":0.486418,"scheduling_latency_s":0.486447,"redis_calls":7,"redis_duration_s":0.001292,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00096,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000332,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019039,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026577,"completed_at":"2026-09-13T10:17:54.760Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002781,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2428gitlab # [ 433.235307] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:54.761Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"ba731f532e1039399dd89ff2","created_at":"2026-09-13T10:17:54.247Z","trace_propagation_headers":{"sentry-trace":"b8edc2c57bb042659310469133d96551-7a9c465ad14f4fcb","baggage":"sentry-trace_id=b8edc2c57bb042659310469133d96551,sentry-release=Unknown"},"correlation_id":"01M2D4BYDZ3XJ64WWR7RP4SE15","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:54.249Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-ba731f532e1039399dd89ff2: start","job_status":"start","queue_duration_s":0.511896,"scheduling_latency_s":0.511924}2429gitlab # [ 433.251563] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:54.791Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"ba731f532e1039399dd89ff2","created_at":"2026-09-13T10:17:54.247Z","trace_propagation_headers":{"sentry-trace":"b8edc2c57bb042659310469133d96551-7a9c465ad14f4fcb","baggage":"sentry-trace_id=b8edc2c57bb042659310469133d96551,sentry-release=Unknown"},"correlation_id":"01M2D4BYDZ3XJ64WWR7RP4SE15","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:54.249Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-ba731f532e1039399dd89ff2: done: 0.029576 sec","job_status":"done","queue_duration_s":0.511896,"scheduling_latency_s":0.511924,"redis_calls":7,"redis_duration_s":0.002557,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000945,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001612,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019594,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029576,"completed_at":"2026-09-13T10:17:54.791Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004446,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2430gitlab # [ 433.293116] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:54.793Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"86da548238ff387c37dd288a","created_at":"2026-09-13T10:17:54.250Z","trace_propagation_headers":{"sentry-trace":"b8edc2c57bb042659310469133d96551-7a9c465ad14f4fcb","baggage":"sentry-trace_id=b8edc2c57bb042659310469133d96551,sentry-release=Unknown"},"correlation_id":"01M2D4BYDZ3XJ64WWR7RP4SE15","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:54.252Z","job_size_bytes":6,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-86da548238ff387c37dd288a: start","job_status":"start","queue_duration_s":0.540982,"scheduling_latency_s":0.541011}2431gitlab # [ 433.308639] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:54.812Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"86da548238ff387c37dd288a","created_at":"2026-09-13T10:17:54.250Z","trace_propagation_headers":{"sentry-trace":"b8edc2c57bb042659310469133d96551-7a9c465ad14f4fcb","baggage":"sentry-trace_id=b8edc2c57bb042659310469133d96551,sentry-release=Unknown"},"correlation_id":"01M2D4BYDZ3XJ64WWR7RP4SE15","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:54.252Z","job_size_bytes":6,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-86da548238ff387c37dd288a: done: 0.019033 sec","job_status":"done","queue_duration_s":0.540982,"scheduling_latency_s":0.541011,"redis_calls":7,"redis_duration_s":0.0037459999999999998,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000944,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002802,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010683,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019033,"completed_at":"2026-09-13T10:17:54.812Z","load_balancing_strategy":"primary","db_duration_s":0.000311,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2432gitlab # [ 433.350034] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:54.814Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"0b2df6c4a7f0bbb82dff1236","created_at":"2026-09-13T10:17:54.253Z","trace_propagation_headers":{"sentry-trace":"b8edc2c57bb042659310469133d96551-7a9c465ad14f4fcb","baggage":"sentry-trace_id=b8edc2c57bb042659310469133d96551,sentry-release=Unknown"},"correlation_id":"01M2D4BYDZ3XJ64WWR7RP4SE15","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:54.255Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-0b2df6c4a7f0bbb82dff1236: start","job_status":"start","queue_duration_s":0.558901,"scheduling_latency_s":0.558926}2433gitlab # [ 433.365467] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:54.839Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"0b2df6c4a7f0bbb82dff1236","created_at":"2026-09-13T10:17:54.253Z","trace_propagation_headers":{"sentry-trace":"b8edc2c57bb042659310469133d96551-7a9c465ad14f4fcb","baggage":"sentry-trace_id=b8edc2c57bb042659310469133d96551,sentry-release=Unknown"},"correlation_id":"01M2D4BYDZ3XJ64WWR7RP4SE15","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:54.255Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-0b2df6c4a7f0bbb82dff1236: done: 0.025315 sec","job_status":"done","queue_duration_s":0.558901,"scheduling_latency_s":0.558926,"redis_calls":7,"redis_duration_s":0.00099,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000696,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000294,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017367,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025315,"completed_at":"2026-09-13T10:17:54.839Z","load_balancing_strategy":"primary","db_duration_s":0.003941,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2434gitlab # [ 433.406308] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:54.841Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"8af4014206d95ffcf98e794f","created_at":"2026-09-13T10:17:54.315Z","trace_propagation_headers":{"sentry-trace":"b8edc2c57bb042659310469133d96551-7a9c465ad14f4fcb","baggage":"sentry-trace_id=b8edc2c57bb042659310469133d96551,sentry-release=Unknown"},"correlation_id":"01M2D4BYDZ3XJ64WWR7RP4SE15","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:54.318Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8af4014206d95ffcf98e794f: start","job_status":"start","queue_duration_s":0.523174,"scheduling_latency_s":0.523194}2435gitlab # [ 433.421980] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:54.929Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"8af4014206d95ffcf98e794f","created_at":"2026-09-13T10:17:54.315Z","trace_propagation_headers":{"sentry-trace":"b8edc2c57bb042659310469133d96551-7a9c465ad14f4fcb","baggage":"sentry-trace_id=b8edc2c57bb042659310469133d96551,sentry-release=Unknown"},"correlation_id":"01M2D4BYDZ3XJ64WWR7RP4SE15","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:54.318Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8af4014206d95ffcf98e794f: done: 0.087549 sec","job_status":"done","queue_duration_s":0.523174,"scheduling_latency_s":0.523194,"redis_calls":8,"redis_duration_s":0.045505,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.042877,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002628,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.015,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021958,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.087549,"completed_at":"2026-09-13T10:17:54.929Z","load_balancing_strategy":"primary","db_duration_s":0.014783,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2436gitlab # [ 433.464296] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:54.932Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"6fa3e20efd3f8c5e0a441e5e","created_at":"2026-09-13T10:17:54.531Z","trace_propagation_headers":{"sentry-trace":"8ccf5089431243d4b900e0730b85a3d1-7643f968603649c1","baggage":"sentry-trace_id=8ccf5089431243d4b900e0730b85a3d1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D4C0AVJF9ZAFSEVE773FYX","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:54.536Z","job_size_bytes":65,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-6fa3e20efd3f8c5e0a441e5e: start","job_status":"start","queue_duration_s":0.39617,"scheduling_latency_s":0.396209}2437gitlab # [ 433.480872] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:55.208Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"6fa3e20efd3f8c5e0a441e5e","created_at":"2026-09-13T10:17:54.531Z","trace_propagation_headers":{"sentry-trace":"8ccf5089431243d4b900e0730b85a3d1-7643f968603649c1","baggage":"sentry-trace_id=8ccf5089431243d4b900e0730b85a3d1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D4C0AVJF9ZAFSEVE773FYX","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:54.536Z","job_size_bytes":65,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-6fa3e20efd3f8c5e0a441e5e: done: 0.276472 sec","job_status":"done","queue_duration_s":0.39617,"scheduling_latency_s":0.396209,"redis_calls":7,"redis_duration_s":0.037498000000000004,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.018477,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.019021,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.07,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.049,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.074008,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.276472,"completed_at":"2026-09-13T10:17:55.208Z","load_balancing_strategy":"primary","db_duration_s":0.094092,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2438gitlab # [ 433.523918] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:55.218Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"6a99aba12a9cf24f218140c9","created_at":"2026-09-13T10:17:54.536Z","trace_propagation_headers":{"sentry-trace":"8ccf5089431243d4b900e0730b85a3d1-7643f968603649c1","baggage":"sentry-trace_id=8ccf5089431243d4b900e0730b85a3d1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D4C0AVJF9ZAFSEVE773FYX","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:54.540Z","job_size_bytes":65,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-6a99aba12a9cf24f218140c9: start","job_status":"start","queue_duration_s":0.678035,"scheduling_latency_s":0.678083}2439gitlab # [ 433.664716] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:55.572Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"6a99aba12a9cf24f218140c9","created_at":"2026-09-13T10:17:54.536Z","trace_propagation_headers":{"sentry-trace":"8ccf5089431243d4b900e0730b85a3d1-7643f968603649c1","baggage":"sentry-trace_id=8ccf5089431243d4b900e0730b85a3d1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D4C0AVJF9ZAFSEVE773FYX","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:54.540Z","job_size_bytes":65,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-6a99aba12a9cf24f218140c9: done: 0.353604 sec","job_status":"done","queue_duration_s":0.678035,"scheduling_latency_s":0.678083,"redis_calls":10,"redis_duration_s":0.019828,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.008226,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.011602,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.152,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.105,"db_ci_duration_s":0.047,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.083127,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.353604,"completed_at":"2026-09-13T10:17:55.572Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.164854,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2440gitlab # [ 433.759380] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:55.594Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"7f40dd810ba80d64493e078e","created_at":"2026-09-13T10:17:54.600Z","trace_propagation_headers":{"sentry-trace":"8ccf5089431243d4b900e0730b85a3d1-7643f968603649c1","baggage":"sentry-trace_id=8ccf5089431243d4b900e0730b85a3d1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D4C0AVJF9ZAFSEVE773FYX","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:54.604Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-7f40dd810ba80d64493e078e: start","job_status":"start","queue_duration_s":0.990104,"scheduling_latency_s":0.99017}2441gitlab # [ 433.859817] gitaly[966]: time="2026-09-13T10:17:55.767Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D4C0AVJF9ZAFSEVE773FYX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-13T10:18:25.765" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:17:55.766" grpc.time_ms=1.522 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2442gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/456921b37007e803dac208d4de47f6e13ba8b9e4/statuses2443gitlab # [ 433.953086] gitaly[966]: time="2026-09-13T10:17:55.861Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4C0AVJF9ZAFSEVE773FYX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:18:25.859" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:17:55.860" grpc.time_ms=1.422 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2444gitlab # [ 433.990455] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:55.893Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"7f40dd810ba80d64493e078e","created_at":"2026-09-13T10:17:54.600Z","trace_propagation_headers":{"sentry-trace":"8ccf5089431243d4b900e0730b85a3d1-7643f968603649c1","baggage":"sentry-trace_id=8ccf5089431243d4b900e0730b85a3d1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D4C0AVJF9ZAFSEVE773FYX","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:54.604Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-7f40dd810ba80d64493e078e: done: 0.299268 sec","job_status":"done","queue_duration_s":0.990104,"scheduling_latency_s":0.99017,"gitaly_calls":2,"gitaly_duration_s":0.012969,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.014696,"redis_read_bytes":80,"redis_write_bytes":5120,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.002098,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.002499,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000518,"redis_queues_read_bytes":3,"redis_queues_write_bytes":1131,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003768,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000464,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.001107,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002144,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.053,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.005,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.01,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.031,"db_ci_duration_s":0.022,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.231075,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.21518626,"duration_s":0.299268,"completed_at":"2026-09-13T10:17:55.893Z","load_balancing_strategy":"primary","db_duration_s":0.057516,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2445gitlab # [ 434.073254] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:55.897Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"e61fc34bb6c3fdc0d70d341d","created_at":"2026-09-13T10:17:54.926Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"369c59d91a14aa2ce88eeb96d3485cd7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:54.943Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-e61fc34bb6c3fdc0d70d341d: start","job_status":"start","queue_duration_s":0.954345,"scheduling_latency_s":0.954391}2446gitlab # [ 434.092131] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:55.926Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}2447gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/456921b37007e803dac208d4de47f6e13ba8b9e4/statuses, in 0.23 seconds)2448gitlab # [ 434.096853] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:55.950Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"e61fc34bb6c3fdc0d70d341d","created_at":"2026-09-13T10:17:54.926Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"369c59d91a14aa2ce88eeb96d3485cd7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:54.943Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-e61fc34bb6c3fdc0d70d341d: done: 0.052225 sec","job_status":"done","queue_duration_s":0.954345,"scheduling_latency_s":0.954391,"redis_calls":9,"redis_duration_s":0.005272,"redis_read_bytes":7,"redis_write_bytes":1178,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000787,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00231,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002175,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024631,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.052225,"completed_at":"2026-09-13T10:17:55.949Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001583,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2449gitlab # [ 434.141670] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:55.954Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"0c2bb830c0dfbe80fcdcc25c","created_at":"2026-09-13T10:17:54.948Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"204a779837e69050b792b831dbc48c6e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-13T10:17:54.956Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-0c2bb830c0dfbe80fcdcc25c: start","job_status":"start","queue_duration_s":0.997959,"scheduling_latency_s":0.998004}2450gitlab # [ 434.154659] gitaly[966]: time="2026-09-13T10:17:55.988Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4C1XTJ58XQ0AWMQV89HFP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:18:25.086" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:17:55.987" grpc.time_ms=0.898 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2451gitlab # [ 434.167545] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:17:56 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/456921b37007e803dac208d4de47f6e13ba8b9e4/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1372452gitlab # [ 434.170317] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:56.024Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"0c2bb830c0dfbe80fcdcc25c","created_at":"2026-09-13T10:17:54.948Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"204a779837e69050b792b831dbc48c6e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-13T10:17:54.956Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-0c2bb830c0dfbe80fcdcc25c: done: 0.07078 sec","job_status":"done","queue_duration_s":0.997959,"scheduling_latency_s":0.998004,"redis_calls":9,"redis_duration_s":0.004189,"redis_read_bytes":7,"redis_write_bytes":1068,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001013,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002168,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001008,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04695,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.07078,"completed_at":"2026-09-13T10:17:56.024Z","load_balancing_strategy":"primary","db_duration_s":0.008335,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2453gitlab # [ 434.210575] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:56.027Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"f473867f7cb81eddf4f060a4","created_at":"2026-09-13T10:17:54.963Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2aa1bc7b84a36fdbc56c48d069f4954a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:54.979Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-f473867f7cb81eddf4f060a4: start","job_status":"start","queue_duration_s":1.048417,"scheduling_latency_s":1.048449}2454gitlab # [ 434.224909] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:56.049Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}2455gitlab # [ 434.228271] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:56.060Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"f473867f7cb81eddf4f060a4","created_at":"2026-09-13T10:17:54.963Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2aa1bc7b84a36fdbc56c48d069f4954a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:54.979Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-f473867f7cb81eddf4f060a4: done: 0.032349 sec","job_status":"done","queue_duration_s":1.048417,"scheduling_latency_s":1.048449,"redis_calls":9,"redis_duration_s":0.0049239999999999996,"redis_read_bytes":7,"redis_write_bytes":1185,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000645,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001651,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002628,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018389,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032349,"completed_at":"2026-09-13T10:17:56.060Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000856,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2456gitlab # [ 434.271213] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:56.062Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"35edcc30dac78c49fe81a83d","created_at":"2026-09-13T10:17:54.992Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2eedaabbb499647800728dcb39510577","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-13T10:17:55.004Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-35edcc30dac78c49fe81a83d: start","job_status":"start","queue_duration_s":1.057673,"scheduling_latency_s":1.057699}2457gitlab # [ 434.284703] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:56.077Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}2458gitlab # [ 434.287943] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:56.086Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"35edcc30dac78c49fe81a83d","created_at":"2026-09-13T10:17:54.992Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2eedaabbb499647800728dcb39510577","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-13T10:17:55.004Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-35edcc30dac78c49fe81a83d: done: 0.02452 sec","job_status":"done","queue_duration_s":1.057673,"scheduling_latency_s":1.057699,"redis_calls":9,"redis_duration_s":0.002533,"redis_read_bytes":7,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00057,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001399,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000564,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013667,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02452,"completed_at":"2026-09-13T10:17:56.086Z","load_balancing_strategy":"primary","db_duration_s":0.000662,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2459gitlab # [ 434.328621] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:56.088Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"8dc8666cc7a65f045f3e83fa","created_at":"2026-09-13T10:17:55.009Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"93f8499035ee470bb04675d5d600b837","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:55.022Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-8dc8666cc7a65f045f3e83fa: start","job_status":"start","queue_duration_s":1.065903,"scheduling_latency_s":1.065925}2460gitlab # [ 434.342764] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:56.112Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"8dc8666cc7a65f045f3e83fa","created_at":"2026-09-13T10:17:55.009Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"93f8499035ee470bb04675d5d600b837","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:55.022Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-8dc8666cc7a65f045f3e83fa: done: 0.024118 sec","job_status":"done","queue_duration_s":1.065903,"scheduling_latency_s":1.065925,"redis_calls":9,"redis_duration_s":0.002515,"redis_read_bytes":7,"redis_write_bytes":1119,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00054,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001439,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000536,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013625,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024118,"completed_at":"2026-09-13T10:17:56.112Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000953,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2461gitlab # [ 434.384532] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:56.114Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"ace683cbe522398342369748","created_at":"2026-09-13T10:17:55.030Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:55.042Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-ace683cbe522398342369748: start","job_status":"start","queue_duration_s":1.07157,"scheduling_latency_s":1.071591}2462gitlab # [ 434.397965] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:56.272Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"ace683cbe522398342369748","created_at":"2026-09-13T10:17:55.030Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:55.042Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-ace683cbe522398342369748: done: 0.158601 sec","job_status":"done","queue_duration_s":1.07157,"scheduling_latency_s":1.071591,"redis_calls":62,"redis_duration_s":0.020578,"redis_read_bytes":110,"redis_write_bytes":41440,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000543,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_calls":2,"redis_queues_duration_s":0.000831,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.018462,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000742,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.138592,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.158601,"completed_at":"2026-09-13T10:17:56.272Z","load_balancing_strategy":"primary","db_duration_s":0.000784,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2463gitlab # [ 434.439938] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:56.274Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"1ca73f171d0becdf856341b5","created_at":"2026-09-13T10:17:55.069Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3200b66ac7573fb76a7a6891c63fcc95","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:55.089Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-1ca73f171d0becdf856341b5: start","job_status":"start","queue_duration_s":1.185182,"scheduling_latency_s":1.185205}2464gitlab # [ 434.454476] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:56.313Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"1ca73f171d0becdf856341b5","created_at":"2026-09-13T10:17:55.069Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3200b66ac7573fb76a7a6891c63fcc95","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:55.089Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-1ca73f171d0becdf856341b5: done: 0.038857 sec","job_status":"done","queue_duration_s":1.185182,"scheduling_latency_s":1.185205,"redis_calls":16,"redis_duration_s":0.004275,"redis_read_bytes":18,"redis_write_bytes":4347,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000524,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_calls":2,"redis_queues_duration_s":0.000375,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002472,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000904,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025999,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.038857,"completed_at":"2026-09-13T10:17:56.313Z","load_balancing_strategy":"primary","db_duration_s":0.002967,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2465gitlab # [ 434.497242] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:56.315Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"7deeaef4f682bfbf3af2bb91","created_at":"2026-09-13T10:17:55.096Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1e46bbe49d169f19531d473004e08337","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:55.110Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-7deeaef4f682bfbf3af2bb91: start","job_status":"start","queue_duration_s":1.204519,"scheduling_latency_s":1.204542}2466gitlab # [ 434.728045] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:56.636Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"7deeaef4f682bfbf3af2bb91","created_at":"2026-09-13T10:17:55.096Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1e46bbe49d169f19531d473004e08337","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:55.110Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-7deeaef4f682bfbf3af2bb91: done: 0.321587 sec","job_status":"done","queue_duration_s":1.204519,"scheduling_latency_s":1.204542,"redis_calls":564,"redis_duration_s":0.063718,"redis_read_bytes":562,"redis_write_bytes":49063,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000513,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001143,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.062062,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.289706,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.321587,"completed_at":"2026-09-13T10:17:56.636Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004138,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2467gitlab # [ 434.822353] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:56.637Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"83675ebf01ffbb74cdb3bb58","created_at":"2026-09-13T10:17:55.119Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c7e91fcfbd3b06dee82639fbce7babf1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-13T10:17:55.133Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-83675ebf01ffbb74cdb3bb58: start","job_status":"start","queue_duration_s":1.504112,"scheduling_latency_s":1.504133}2468gitlab # [ 434.839310] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:56.646Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}2469gitlab # [ 434.843607] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:56.653Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"83675ebf01ffbb74cdb3bb58","created_at":"2026-09-13T10:17:55.119Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c7e91fcfbd3b06dee82639fbce7babf1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-13T10:17:55.133Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-83675ebf01ffbb74cdb3bb58: done: 0.015456 sec","job_status":"done","queue_duration_s":1.504112,"scheduling_latency_s":1.504133,"redis_calls":8,"redis_duration_s":0.001513,"redis_read_bytes":133,"redis_write_bytes":908,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000153,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":107,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000939,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000421,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008175,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015456,"completed_at":"2026-09-13T10:17:56.653Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2470gitlab # [ 434.886849] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:56.654Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"514d841953c56244c21eb9a9","created_at":"2026-09-13T10:17:55.167Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"40e6edb37f3803ea3eef6ac0b6b2859d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:55.181Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-514d841953c56244c21eb9a9: start","job_status":"start","queue_duration_s":1.47348,"scheduling_latency_s":1.473503}2471gitlab # [ 434.929448] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:56.696Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"514d841953c56244c21eb9a9","created_at":"2026-09-13T10:17:55.167Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"40e6edb37f3803ea3eef6ac0b6b2859d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:55.181Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-514d841953c56244c21eb9a9: done: 0.041991 sec","job_status":"done","queue_duration_s":1.47348,"scheduling_latency_s":1.473503,"redis_calls":9,"redis_duration_s":0.002341,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001223,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001118,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.017,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018819,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00055012,"exclusive_lock_hold_duration_s":0.02758776,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.041991,"completed_at":"2026-09-13T10:17:56.696Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.018625,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2472gitlab # [ 434.996030] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:56.698Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"8c20bc4e6e78f8c6ee4f5b04","created_at":"2026-09-13T10:17:55.203Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dca7755ff986def1a5bd56b98fee1c83","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:55.213Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-8c20bc4e6e78f8c6ee4f5b04: start","job_status":"start","queue_duration_s":1.485231,"scheduling_latency_s":1.485248}2473gitlab # [ 435.011339] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:56.705Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}2474gitlab # [ 435.014688] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:56.710Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"8c20bc4e6e78f8c6ee4f5b04","created_at":"2026-09-13T10:17:55.203Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dca7755ff986def1a5bd56b98fee1c83","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:55.213Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-8c20bc4e6e78f8c6ee4f5b04: done: 0.012396 sec","job_status":"done","queue_duration_s":1.485231,"scheduling_latency_s":1.485248,"redis_calls":7,"redis_duration_s":0.0016049999999999999,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001148,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000457,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007373,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012396,"completed_at":"2026-09-13T10:17:56.710Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2475gitlab # [ 435.054800] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:56.712Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"0219cf1c761317a1f59bf689","created_at":"2026-09-13T10:17:55.264Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c53eabd6d13bf5e6255666e5d56755de","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:55.269Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-0219cf1c761317a1f59bf689: start","job_status":"start","queue_duration_s":1.442979,"scheduling_latency_s":1.442995}2476gitlab # [ 435.069334] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:56.724Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"0219cf1c761317a1f59bf689","created_at":"2026-09-13T10:17:55.264Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c53eabd6d13bf5e6255666e5d56755de","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:55.269Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-0219cf1c761317a1f59bf689: done: 0.012624 sec","job_status":"done","queue_duration_s":1.442979,"scheduling_latency_s":1.442995,"redis_calls":7,"redis_duration_s":0.001602,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001156,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000446,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007669,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012624,"completed_at":"2026-09-13T10:17:56.724Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2477gitlab # [ 435.110277] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:56.726Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"7ce3a842f9c74f39f0ed84fd","created_at":"2026-09-13T10:17:55.280Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"44500906021e0d9bad690d7a316dedff","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:55.284Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-7ce3a842f9c74f39f0ed84fd: start","job_status":"start","queue_duration_s":1.441941,"scheduling_latency_s":1.441957}2478gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/456921b37007e803dac208d4de47f6e13ba8b9e4/statuses2479gitlab # [ 435.124328] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:56.748Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"7ce3a842f9c74f39f0ed84fd","created_at":"2026-09-13T10:17:55.280Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"44500906021e0d9bad690d7a316dedff","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:55.284Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-7ce3a842f9c74f39f0ed84fd: done: 0.022124 sec","job_status":"done","queue_duration_s":1.441941,"scheduling_latency_s":1.441957,"redis_calls":7,"redis_duration_s":0.001826,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001325,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000501,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010853,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022124,"completed_at":"2026-09-13T10:17:56.748Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006636,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2480gitlab # [ 435.162158] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:56.750Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"9efaac62ece8f2ee87206cb7","created_at":"2026-09-13T10:17:55.295Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"70392c13678238572b6337e8b6799730","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:55.307Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-9efaac62ece8f2ee87206cb7: start","job_status":"start","queue_duration_s":1.442536,"scheduling_latency_s":1.442557}2481gitlab # [ 435.175375] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:56.768Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"9efaac62ece8f2ee87206cb7","created_at":"2026-09-13T10:17:55.295Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"70392c13678238572b6337e8b6799730","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:55.307Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-9efaac62ece8f2ee87206cb7: done: 0.017779 sec","job_status":"done","queue_duration_s":1.442536,"scheduling_latency_s":1.442557,"redis_calls":9,"redis_duration_s":0.002281,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001213,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001068,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01068,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00053108,"exclusive_lock_hold_duration_s":0.00399688,"duration_s":0.017779,"completed_at":"2026-09-13T10:17:56.767Z","load_balancing_strategy":"primary","db_duration_s":0.001609,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2482gitlab # [ 435.215968] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:56.769Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"c1cd4c5075eba0216cf4ea0a","created_at":"2026-09-13T10:17:55.325Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a33c7fd800ce12082cfff2d3b48af48c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:55.329Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-c1cd4c5075eba0216cf4ea0a: start","job_status":"start","queue_duration_s":1.439997,"scheduling_latency_s":1.440013}2483gitlab # [ 435.229679] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.087Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"c1cd4c5075eba0216cf4ea0a","created_at":"2026-09-13T10:17:55.325Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a33c7fd800ce12082cfff2d3b48af48c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:55.329Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-c1cd4c5075eba0216cf4ea0a: done: 0.318166 sec","job_status":"done","queue_duration_s":1.439997,"scheduling_latency_s":1.440013,"redis_calls":9,"redis_duration_s":0.005922,"redis_read_bytes":7,"redis_write_bytes":987,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004632,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00129,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":129,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":129,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.053,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.053,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.245586,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0005596,"exclusive_lock_hold_duration_s":0.2960254,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.318166,"completed_at":"2026-09-13T10:17:57.087Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.084177,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2484gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/456921b37007e803dac208d4de47f6e13ba8b9e4/statuses, in 0.15 seconds)2485gitlab # [ 435.271697] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.089Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"fbab82625b134ddac6d7f614","created_at":"2026-09-13T10:17:55.348Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5db08a522a50891d0f4c9718da1a2368","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:55.352Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-fbab82625b134ddac6d7f614: start","job_status":"start","queue_duration_s":1.737287,"scheduling_latency_s":1.73731}2486gitlab # [ 435.286114] gitaly[966]: time="2026-09-13T10:17:57.157Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4C33N5Q4Z9J92AN4G1QY3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:18:27.155" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:17:57.156" grpc.time_ms=1.083 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2487gitlab # [ 435.299846] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:17:57 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/456921b37007e803dac208d4de47f6e13ba8b9e4/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 762488gitlab # [ 435.302586] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.125Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"fbab82625b134ddac6d7f614","created_at":"2026-09-13T10:17:55.348Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5db08a522a50891d0f4c9718da1a2368","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:55.352Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-fbab82625b134ddac6d7f614: done: 0.035309 sec","job_status":"done","queue_duration_s":1.737287,"scheduling_latency_s":1.73731,"redis_calls":7,"redis_duration_s":0.0037219999999999996,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002708,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001014,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024745,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.035309,"completed_at":"2026-09-13T10:17:57.125Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00205,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2489gitlab # [ 435.342433] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.129Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"ed476a72cefc24028497db77","created_at":"2026-09-13T10:17:55.385Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0d16bfa7effbf4f712f6e092d6cf9f56","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:55.393Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-ed476a72cefc24028497db77: start","job_status":"start","queue_duration_s":1.735646,"scheduling_latency_s":1.735728}2490gitlab # [ 435.356314] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.180Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"ed476a72cefc24028497db77","created_at":"2026-09-13T10:17:55.385Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0d16bfa7effbf4f712f6e092d6cf9f56","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:55.393Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-ed476a72cefc24028497db77: done: 0.051183 sec","job_status":"done","queue_duration_s":1.735646,"scheduling_latency_s":1.735728,"redis_calls":7,"redis_duration_s":0.0028049999999999998,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00207,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000735,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021644,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.051183,"completed_at":"2026-09-13T10:17:57.180Z","load_balancing_strategy":"primary","db_duration_s":0.01802,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2491gitlab # [ 435.397000] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.184Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"f99c05afcb656b9d23d4abf5","created_at":"2026-09-13T10:17:55.399Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"80d9e3b1ff4ff4d962a76a53844b4e63","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:55.410Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-f99c05afcb656b9d23d4abf5: start","job_status":"start","queue_duration_s":1.774221,"scheduling_latency_s":1.774265}2492gitlab # [ 435.411683] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.259Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"f99c05afcb656b9d23d4abf5","created_at":"2026-09-13T10:17:55.399Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"80d9e3b1ff4ff4d962a76a53844b4e63","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:55.410Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-f99c05afcb656b9d23d4abf5: done: 0.075115 sec","job_status":"done","queue_duration_s":1.774221,"scheduling_latency_s":1.774265,"redis_calls":9,"redis_duration_s":0.004506,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002408,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002098,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040041,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00142062,"exclusive_lock_hold_duration_s":0.0288302,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.075115,"completed_at":"2026-09-13T10:17:57.259Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.024864,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2493gitlab # [ 435.453972] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.263Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"5ec65d698ea576e26ca37c5d","created_at":"2026-09-13T10:17:55.422Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"49df68a2e96a6a5d0784e6974616c8be","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:55.430Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-5ec65d698ea576e26ca37c5d: start","job_status":"start","queue_duration_s":1.833362,"scheduling_latency_s":1.833405}2494gitlab # [ 435.468137] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.295Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"5ec65d698ea576e26ca37c5d","created_at":"2026-09-13T10:17:55.422Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"49df68a2e96a6a5d0784e6974616c8be","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:55.430Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-5ec65d698ea576e26ca37c5d: done: 0.032566 sec","job_status":"done","queue_duration_s":1.833362,"scheduling_latency_s":1.833405,"redis_calls":9,"redis_duration_s":0.003299,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001786,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001513,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0201,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00091852,"exclusive_lock_hold_duration_s":0.0051385,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.032566,"completed_at":"2026-09-13T10:17:57.295Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002089,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2495gitlab # [ 435.510267] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.298Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"71238d30602cf6fd7d4481a9","created_at":"2026-09-13T10:17:55.451Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8b3569a825726acee20579b61ffcba92","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:55.460Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-71238d30602cf6fd7d4481a9: start","job_status":"start","queue_duration_s":1.837648,"scheduling_latency_s":1.837678}2496gitlab # [ 435.524702] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.343Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"71238d30602cf6fd7d4481a9","created_at":"2026-09-13T10:17:55.451Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8b3569a825726acee20579b61ffcba92","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:55.460Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-71238d30602cf6fd7d4481a9: done: 0.045271 sec","job_status":"done","queue_duration_s":1.837648,"scheduling_latency_s":1.837678,"redis_calls":10,"redis_duration_s":0.005936,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002588,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003348,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030751,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01033814,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.045271,"completed_at":"2026-09-13T10:17:57.343Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003651,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2497gitlab # [ 435.566885] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.348Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"a2b7604393255c24fc677f17","created_at":"2026-09-13T10:17:55.485Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3f9ce3d274a462b2d227eb154c3baf83","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:55.492Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-a2b7604393255c24fc677f17: start","job_status":"start","queue_duration_s":1.855668,"scheduling_latency_s":1.855714}2498gitlab # [ 435.581313] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.367Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}2499gitlab # [ 435.584885] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.378Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"a2b7604393255c24fc677f17","created_at":"2026-09-13T10:17:55.485Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3f9ce3d274a462b2d227eb154c3baf83","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:55.492Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-a2b7604393255c24fc677f17: done: 0.031062 sec","job_status":"done","queue_duration_s":1.855668,"scheduling_latency_s":1.855714,"redis_calls":7,"redis_duration_s":0.0027990000000000003,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002053,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000746,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019966,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031062,"completed_at":"2026-09-13T10:17:57.378Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2500gitlab # [ 435.624956] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.381Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ArchiveTracesCronWorker","jid":"b174d4d817901fcaf2e61796","created_at":"2026-09-13T10:17:55.539Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cfb3ef14e21c9cd0e0d2c963b8146b4c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f0234f524ae629146c97646d3c809b290751f669b415a8b01cafe8f68b01506d","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:55.564Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ArchiveTracesCronWorker JID-b174d4d817901fcaf2e61796: start","job_status":"start","queue_duration_s":1.816966,"scheduling_latency_s":1.816997}2501gitlab # [ 435.638104] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.434Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ArchiveTracesCronWorker","jid":"b174d4d817901fcaf2e61796","created_at":"2026-09-13T10:17:55.539Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-e461810b05bd47c0","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cfb3ef14e21c9cd0e0d2c963b8146b4c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f0234f524ae629146c97646d3c809b290751f669b415a8b01cafe8f68b01506d","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:55.564Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ArchiveTracesCronWorker JID-b174d4d817901fcaf2e61796: done: 0.05292 sec","job_status":"done","queue_duration_s":1.816966,"scheduling_latency_s":1.816997,"redis_calls":12,"redis_duration_s":0.004461,"redis_read_bytes":11,"redis_write_bytes":1354,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00076,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":297,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001901,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.0018,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":519,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032828,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0008915,"exclusive_lock_hold_duration_s":0.01738284,"duration_s":0.05292,"completed_at":"2026-09-13T10:17:57.434Z","load_balancing_strategy":"primary","db_duration_s":0.006389,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2502gitlab # [ 435.679175] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.436Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"1d7a730862fbda1bb79ef909","created_at":"2026-09-13T10:17:55.732Z","trace_propagation_headers":{"sentry-trace":"8ccf5089431243d4b900e0730b85a3d1-dccd0a54e77c4f8d","baggage":"sentry-trace_id=8ccf5089431243d4b900e0730b85a3d1,sentry-release=Unknown"},"correlation_id":"01M2D4C0AVJF9ZAFSEVE773FYX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:55.737Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-1d7a730862fbda1bb79ef909: start","job_status":"start","queue_duration_s":1.699049,"scheduling_latency_s":1.699078}2503gitlab # [ 435.695507] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.497Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"1d7a730862fbda1bb79ef909","created_at":"2026-09-13T10:17:55.732Z","trace_propagation_headers":{"sentry-trace":"8ccf5089431243d4b900e0730b85a3d1-dccd0a54e77c4f8d","baggage":"sentry-trace_id=8ccf5089431243d4b900e0730b85a3d1,sentry-release=Unknown"},"correlation_id":"01M2D4C0AVJF9ZAFSEVE773FYX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-13T10:17:55.737Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-1d7a730862fbda1bb79ef909: done: 0.061145 sec","job_status":"done","queue_duration_s":1.699049,"scheduling_latency_s":1.699078,"redis_calls":7,"redis_duration_s":0.002303,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001684,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000619,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040824,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.061145,"completed_at":"2026-09-13T10:17:57.497Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014583,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2504gitlab # [ 435.739179] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.500Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7b33d0e4cd1f9bd430222dc8","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7b33d0e4cd1f9bd430222dc8: start","job_status":"start","queue_duration_s":1.238687,"scheduling_latency_s":1.238717}2505gitlab # [ 435.751662] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.523Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7b33d0e4cd1f9bd430222dc8","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7b33d0e4cd1f9bd430222dc8: done: 0.023224 sec","job_status":"done","queue_duration_s":1.238687,"scheduling_latency_s":1.238717,"redis_calls":13,"redis_duration_s":0.003464,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001098,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002366,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014918,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023224,"completed_at":"2026-09-13T10:17:57.523Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2506gitlab # [ 435.790796] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.525Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d557cf4f2d3cc474489d3620","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d557cf4f2d3cc474489d3620: start","job_status":"start","queue_duration_s":1.263713,"scheduling_latency_s":1.263735}2507gitlab # [ 435.803230] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.544Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d557cf4f2d3cc474489d3620","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d557cf4f2d3cc474489d3620: done: 0.01943 sec","job_status":"done","queue_duration_s":1.263713,"scheduling_latency_s":1.263735,"redis_calls":13,"redis_duration_s":0.003142,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001027,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002115,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012282,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01943,"completed_at":"2026-09-13T10:17:57.544Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2508gitlab # [ 435.842410] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.546Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"86f6072196ce75500a7061cb","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-86f6072196ce75500a7061cb: start","job_status":"start","queue_duration_s":1.284752,"scheduling_latency_s":1.28477}2509gitlab # [ 435.854841] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.562Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"86f6072196ce75500a7061cb","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-86f6072196ce75500a7061cb: done: 0.016647 sec","job_status":"done","queue_duration_s":1.284752,"scheduling_latency_s":1.28477,"redis_calls":13,"redis_duration_s":0.002783,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000996,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001787,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010502,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016647,"completed_at":"2026-09-13T10:17:57.562Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2510gitlab # [ 435.893829] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.564Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"211f09545244c1f943f68f4b","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-211f09545244c1f943f68f4b: start","job_status":"start","queue_duration_s":1.302889,"scheduling_latency_s":1.302906}2511gitlab # [ 435.907246] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.579Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"211f09545244c1f943f68f4b","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-211f09545244c1f943f68f4b: done: 0.015494 sec","job_status":"done","queue_duration_s":1.302889,"scheduling_latency_s":1.302906,"redis_calls":13,"redis_duration_s":0.002855,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000969,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001886,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009495,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015494,"completed_at":"2026-09-13T10:17:57.579Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2512gitlab # [ 435.946053] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.581Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a742402d4828e6cf7093823","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a742402d4828e6cf7093823: start","job_status":"start","queue_duration_s":1.319775,"scheduling_latency_s":1.319793}2513gitlab # [ 435.958421] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.596Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a742402d4828e6cf7093823","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a742402d4828e6cf7093823: done: 0.015827 sec","job_status":"done","queue_duration_s":1.319775,"scheduling_latency_s":1.319793,"redis_calls":13,"redis_duration_s":0.00324,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001411,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001829,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00966,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015827,"completed_at":"2026-09-13T10:17:57.596Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2514gitlab # [ 435.997311] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.598Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dd7259e7e657adfc509f827a","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dd7259e7e657adfc509f827a: start","job_status":"start","queue_duration_s":1.337084,"scheduling_latency_s":1.337102}2515gitlab # [ 436.009638] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.625Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dd7259e7e657adfc509f827a","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dd7259e7e657adfc509f827a: done: 0.026645 sec","job_status":"done","queue_duration_s":1.337084,"scheduling_latency_s":1.337102,"redis_calls":13,"redis_duration_s":0.004491,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001422,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003069,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018532,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026645,"completed_at":"2026-09-13T10:17:57.625Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2516gitlab # [ 436.048468] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.628Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"48bc1dc6ca85cb9369d4205c","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-48bc1dc6ca85cb9369d4205c: start","job_status":"start","queue_duration_s":1.367251,"scheduling_latency_s":1.367296}2517gitlab # [ 436.060696] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.667Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"48bc1dc6ca85cb9369d4205c","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-48bc1dc6ca85cb9369d4205c: done: 0.039167 sec","job_status":"done","queue_duration_s":1.367251,"scheduling_latency_s":1.367296,"redis_calls":13,"redis_duration_s":0.004603,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001603,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027234,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.039167,"completed_at":"2026-09-13T10:17:57.667Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2518gitlab # [ 436.099447] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.670Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"75e367701a5c1cd1ae778cff","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-75e367701a5c1cd1ae778cff: start","job_status":"start","queue_duration_s":1.408949,"scheduling_latency_s":1.408988}2519gitlab # [ 436.111910] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.698Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"75e367701a5c1cd1ae778cff","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-75e367701a5c1cd1ae778cff: done: 0.028279 sec","job_status":"done","queue_duration_s":1.408949,"scheduling_latency_s":1.408988,"redis_calls":13,"redis_duration_s":0.0043289999999999995,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001281,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003048,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017812,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028279,"completed_at":"2026-09-13T10:17:57.698Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2520gitlab # [ 436.151933] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.701Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d60298bf7ae05e67b299aff","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d60298bf7ae05e67b299aff: start","job_status":"start","queue_duration_s":1.439583,"scheduling_latency_s":1.43962}2521gitlab # [ 436.164448] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.742Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d60298bf7ae05e67b299aff","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d60298bf7ae05e67b299aff: done: 0.041192 sec","job_status":"done","queue_duration_s":1.439583,"scheduling_latency_s":1.43962,"redis_calls":13,"redis_duration_s":0.005393,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001857,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003536,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028864,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.041192,"completed_at":"2026-09-13T10:17:57.742Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2522gitlab # [ 436.203393] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.745Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d35b49e9868c16bbc89024f7","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d35b49e9868c16bbc89024f7: start","job_status":"start","queue_duration_s":1.483691,"scheduling_latency_s":1.483733}2523gitlab # [ 436.215841] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.773Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d35b49e9868c16bbc89024f7","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d35b49e9868c16bbc89024f7: done: 0.028345 sec","job_status":"done","queue_duration_s":1.483691,"scheduling_latency_s":1.483733,"redis_calls":13,"redis_duration_s":0.0037029999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001293,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00241,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017955,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028345,"completed_at":"2026-09-13T10:17:57.773Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2524gitlab # [ 436.254582] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.775Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"489e4ad3ac4aca2d6a970d0e","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-489e4ad3ac4aca2d6a970d0e: start","job_status":"start","queue_duration_s":1.514354,"scheduling_latency_s":1.514386}2525gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/456921b37007e803dac208d4de47f6e13ba8b9e4/statuses2526gitlab # [ 436.266955] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.799Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"489e4ad3ac4aca2d6a970d0e","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-489e4ad3ac4aca2d6a970d0e: done: 0.024194 sec","job_status":"done","queue_duration_s":1.514354,"scheduling_latency_s":1.514386,"redis_calls":13,"redis_duration_s":0.004018,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001147,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002871,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015221,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024194,"completed_at":"2026-09-13T10:17:57.799Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2527gitlab # [ 436.304257] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.802Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34e2a73e29a782f3e6b18a90","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34e2a73e29a782f3e6b18a90: start","job_status":"start","queue_duration_s":1.540632,"scheduling_latency_s":1.540664}2528gitlab # [ 436.315412] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.824Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34e2a73e29a782f3e6b18a90","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34e2a73e29a782f3e6b18a90: done: 0.022301 sec","job_status":"done","queue_duration_s":1.540632,"scheduling_latency_s":1.540664,"redis_calls":13,"redis_duration_s":0.003307,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001155,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002152,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014164,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022301,"completed_at":"2026-09-13T10:17:57.824Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2529gitlab # [ 436.354166] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.826Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"205ca615b8f434893bf7c4f6","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-205ca615b8f434893bf7c4f6: start","job_status":"start","queue_duration_s":1.565024,"scheduling_latency_s":1.565053}2530gitlab # [ 436.366574] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.846Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"205ca615b8f434893bf7c4f6","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-205ca615b8f434893bf7c4f6: done: 0.020107 sec","job_status":"done","queue_duration_s":1.565024,"scheduling_latency_s":1.565053,"redis_calls":13,"redis_duration_s":0.0030499999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001064,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001986,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012573,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020107,"completed_at":"2026-09-13T10:17:57.846Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2531gitlab # [ 436.405584] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.848Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"37478faa030296b8be97d6cb","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-37478faa030296b8be97d6cb: start","job_status":"start","queue_duration_s":1.587077,"scheduling_latency_s":1.587106}2532gitlab # [ 436.417935] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.867Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"37478faa030296b8be97d6cb","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-37478faa030296b8be97d6cb: done: 0.019245 sec","job_status":"done","queue_duration_s":1.587077,"scheduling_latency_s":1.587106,"redis_calls":13,"redis_duration_s":0.003337,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00102,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002317,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011686,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019245,"completed_at":"2026-09-13T10:17:57.867Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2533gitlab # [ 436.456752] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.869Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7d1b12031fcdde6302867c2e","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7d1b12031fcdde6302867c2e: start","job_status":"start","queue_duration_s":1.608078,"scheduling_latency_s":1.608106}2534gitlab # [ 436.469026] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.887Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7d1b12031fcdde6302867c2e","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7d1b12031fcdde6302867c2e: done: 0.01844 sec","job_status":"done","queue_duration_s":1.608078,"scheduling_latency_s":1.608106,"redis_calls":13,"redis_duration_s":0.003094,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001062,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002032,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011202,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01844,"completed_at":"2026-09-13T10:17:57.887Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2535gitlab # [ 436.507679] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.889Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"694c41e76a0b7ba963687ca4","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-694c41e76a0b7ba963687ca4: start","job_status":"start","queue_duration_s":1.628162,"scheduling_latency_s":1.628189}2536gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/456921b37007e803dac208d4de47f6e13ba8b9e4/statuses, in 0.25 seconds)2537gitlab # [ 436.520234] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.906Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"694c41e76a0b7ba963687ca4","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-694c41e76a0b7ba963687ca4: done: 0.017284 sec","job_status":"done","queue_duration_s":1.628162,"scheduling_latency_s":1.628189,"redis_calls":13,"redis_duration_s":0.003012,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00099,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002022,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010313,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017284,"completed_at":"2026-09-13T10:17:57.906Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2538gitlab # [ 436.559538] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.908Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"550ae491e82eb72300a35b9f","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-550ae491e82eb72300a35b9f: start","job_status":"start","queue_duration_s":1.647168,"scheduling_latency_s":1.647195}2539gitlab # [ 436.572249] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:17:58 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/456921b37007e803dac208d4de47f6e13ba8b9e4/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1402540gitlab # [ 436.575123] gitaly[966]: time="2026-09-13T10:17:58.386Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4C48EX97CFYV0F1DT8AR5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:18:28.383" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:17:58.384" grpc.time_ms=1.153 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2541gitlab # [ 436.587947] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.925Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"550ae491e82eb72300a35b9f","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-550ae491e82eb72300a35b9f: done: 0.01666 sec","job_status":"done","queue_duration_s":1.647168,"scheduling_latency_s":1.647195,"redis_calls":13,"redis_duration_s":0.002823,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000973,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00185,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010052,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01666,"completed_at":"2026-09-13T10:17:57.925Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2542gitlab # [ 436.626806] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.926Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"13f66ca78815769a2cf04b4b","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-13f66ca78815769a2cf04b4b: start","job_status":"start","queue_duration_s":1.665454,"scheduling_latency_s":1.665476}2543gitlab # [ 436.639156] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.941Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"13f66ca78815769a2cf04b4b","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-13f66ca78815769a2cf04b4b: done: 0.015187 sec","job_status":"done","queue_duration_s":1.665454,"scheduling_latency_s":1.665476,"redis_calls":13,"redis_duration_s":0.002571,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000946,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001625,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009253,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015187,"completed_at":"2026-09-13T10:17:57.941Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2544gitlab # [ 436.677447] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.943Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aa1023cac87c7e4c6a16122c","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aa1023cac87c7e4c6a16122c: start","job_status":"start","queue_duration_s":1.682098,"scheduling_latency_s":1.682115}2545gitlab # [ 436.689382] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.958Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aa1023cac87c7e4c6a16122c","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aa1023cac87c7e4c6a16122c: done: 0.014939 sec","job_status":"done","queue_duration_s":1.682098,"scheduling_latency_s":1.682115,"redis_calls":13,"redis_duration_s":0.002733,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000918,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001815,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009209,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014939,"completed_at":"2026-09-13T10:17:57.958Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2546gitlab # [ 436.727760] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.959Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ac8e5570f1a142cba76e0eda","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ac8e5570f1a142cba76e0eda: start","job_status":"start","queue_duration_s":1.698428,"scheduling_latency_s":1.698445}2547gitlab # [ 436.741010] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.974Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ac8e5570f1a142cba76e0eda","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ac8e5570f1a142cba76e0eda: done: 0.014771 sec","job_status":"done","queue_duration_s":1.698428,"scheduling_latency_s":1.698445,"redis_calls":13,"redis_duration_s":0.0025009999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000926,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001575,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009285,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014771,"completed_at":"2026-09-13T10:17:57.974Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2548gitlab # [ 436.779830] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.976Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"795337c517ee8f59eb14312c","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-795337c517ee8f59eb14312c: start","job_status":"start","queue_duration_s":1.71465,"scheduling_latency_s":1.714665}2549gitlab # [ 436.792040] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.990Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"795337c517ee8f59eb14312c","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-795337c517ee8f59eb14312c: done: 0.014796 sec","job_status":"done","queue_duration_s":1.71465,"scheduling_latency_s":1.714665,"redis_calls":13,"redis_duration_s":0.002561,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000912,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001649,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009298,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014796,"completed_at":"2026-09-13T10:17:57.990Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2550gitlab # [ 436.830293] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:57.992Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4e75d978d7d2c2ba0ff984bc","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4e75d978d7d2c2ba0ff984bc: start","job_status":"start","queue_duration_s":1.731425,"scheduling_latency_s":1.731443}2551gitlab # [ 436.842876] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.007Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4e75d978d7d2c2ba0ff984bc","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4e75d978d7d2c2ba0ff984bc: done: 0.014995 sec","job_status":"done","queue_duration_s":1.731425,"scheduling_latency_s":1.731443,"redis_calls":13,"redis_duration_s":0.002765,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000929,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001836,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009228,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014995,"completed_at":"2026-09-13T10:17:58.007Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2552gitlab # [ 436.881682] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.009Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"03d2243c459c4e6d29ca16bd","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-03d2243c459c4e6d29ca16bd: start","job_status":"start","queue_duration_s":1.747864,"scheduling_latency_s":1.74788}2553gitlab # [ 436.893938] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.024Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"03d2243c459c4e6d29ca16bd","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-03d2243c459c4e6d29ca16bd: done: 0.015069 sec","job_status":"done","queue_duration_s":1.747864,"scheduling_latency_s":1.74788,"redis_calls":13,"redis_duration_s":0.002667,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000965,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001702,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009489,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015069,"completed_at":"2026-09-13T10:17:58.024Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2554gitlab # [ 436.932240] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.025Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6335c5f1d7f975e2582cc88b","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6335c5f1d7f975e2582cc88b: start","job_status":"start","queue_duration_s":1.764387,"scheduling_latency_s":1.764404}2555gitlab # [ 436.945128] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.040Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6335c5f1d7f975e2582cc88b","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6335c5f1d7f975e2582cc88b: done: 0.014959 sec","job_status":"done","queue_duration_s":1.764387,"scheduling_latency_s":1.764404,"redis_calls":13,"redis_duration_s":0.00273,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000933,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001797,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009241,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014959,"completed_at":"2026-09-13T10:17:58.040Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2556gitlab # [ 436.983699] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.042Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7ca39c84308309d9c32655c4","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7ca39c84308309d9c32655c4: start","job_status":"start","queue_duration_s":1.780629,"scheduling_latency_s":1.780646}2557gitlab # [ 436.995963] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.056Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7ca39c84308309d9c32655c4","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7ca39c84308309d9c32655c4: done: 0.014665 sec","job_status":"done","queue_duration_s":1.780629,"scheduling_latency_s":1.780646,"redis_calls":13,"redis_duration_s":0.002582,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000933,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001649,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009156,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014665,"completed_at":"2026-09-13T10:17:58.056Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2558gitlab # [ 437.034329] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.057Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9086390bcad5631beb82b08a","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9086390bcad5631beb82b08a: start","job_status":"start","queue_duration_s":1.796585,"scheduling_latency_s":1.796602}2559gitlab # [ 437.046547] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.072Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9086390bcad5631beb82b08a","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9086390bcad5631beb82b08a: done: 0.014829 sec","job_status":"done","queue_duration_s":1.796585,"scheduling_latency_s":1.796602,"redis_calls":13,"redis_duration_s":0.002679,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000908,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001771,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009217,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014829,"completed_at":"2026-09-13T10:17:58.072Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2560gitlab # [ 437.084477] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.074Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e82d43c38caa08d53821d4a","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e82d43c38caa08d53821d4a: start","job_status":"start","queue_duration_s":1.812731,"scheduling_latency_s":1.812747}2561gitlab # [ 437.096475] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.088Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e82d43c38caa08d53821d4a","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e82d43c38caa08d53821d4a: done: 0.014888 sec","job_status":"done","queue_duration_s":1.812731,"scheduling_latency_s":1.812747,"redis_calls":13,"redis_duration_s":0.00283,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000971,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001859,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009184,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014888,"completed_at":"2026-09-13T10:17:58.088Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2562gitlab # [ 437.134388] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.090Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"393770009830cf4f3f6f30f4","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-393770009830cf4f3f6f30f4: start","job_status":"start","queue_duration_s":1.829037,"scheduling_latency_s":1.829054}2563gitlab # [ 437.147409] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.106Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"393770009830cf4f3f6f30f4","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-393770009830cf4f3f6f30f4: done: 0.015662 sec","job_status":"done","queue_duration_s":1.829037,"scheduling_latency_s":1.829054,"redis_calls":13,"redis_duration_s":0.0028269999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001035,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001792,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009355,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015662,"completed_at":"2026-09-13T10:17:58.106Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2564gitlab # [ 437.186190] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.107Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"31d6c89195fe52b5c8e6eaf1","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-31d6c89195fe52b5c8e6eaf1: start","job_status":"start","queue_duration_s":1.846153,"scheduling_latency_s":1.846169}2565gitlab # [ 437.198370] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.122Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"31d6c89195fe52b5c8e6eaf1","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-31d6c89195fe52b5c8e6eaf1: done: 0.015116 sec","job_status":"done","queue_duration_s":1.846153,"scheduling_latency_s":1.846169,"redis_calls":13,"redis_duration_s":0.002907,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000979,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001928,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00935,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015116,"completed_at":"2026-09-13T10:17:58.122Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2566gitlab # [ 437.237154] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.124Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ce64c8e4616621c88e9ca74d","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ce64c8e4616621c88e9ca74d: start","job_status":"start","queue_duration_s":1.863008,"scheduling_latency_s":1.863025}2567gitlab # [ 437.249270] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.139Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ce64c8e4616621c88e9ca74d","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ce64c8e4616621c88e9ca74d: done: 0.015146 sec","job_status":"done","queue_duration_s":1.863008,"scheduling_latency_s":1.863025,"redis_calls":13,"redis_duration_s":0.002953,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001013,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00194,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009108,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015146,"completed_at":"2026-09-13T10:17:58.139Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2568gitlab # [ 437.287811] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.140Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fee2f5286429dba854d89ae0","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fee2f5286429dba854d89ae0: start","job_status":"start","queue_duration_s":1.879588,"scheduling_latency_s":1.879604}2569gitlab # [ 437.300084] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.155Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fee2f5286429dba854d89ae0","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fee2f5286429dba854d89ae0: done: 0.014748 sec","job_status":"done","queue_duration_s":1.879588,"scheduling_latency_s":1.879604,"redis_calls":13,"redis_duration_s":0.002721,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000922,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001799,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009094,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014748,"completed_at":"2026-09-13T10:17:58.155Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2570gitlab # [ 437.338332] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.157Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"991aa80b0015280ba5277b6a","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-991aa80b0015280ba5277b6a: start","job_status":"start","queue_duration_s":1.895647,"scheduling_latency_s":1.895662}2571gitlab # [ 437.350567] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.172Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"991aa80b0015280ba5277b6a","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-991aa80b0015280ba5277b6a: done: 0.015851 sec","job_status":"done","queue_duration_s":1.895647,"scheduling_latency_s":1.895662,"redis_calls":13,"redis_duration_s":0.002676,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000939,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001737,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009673,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015851,"completed_at":"2026-09-13T10:17:58.172Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2572gitlab # [ 437.389839] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.174Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5bf0519f4faa8aadc0f83d19","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5bf0519f4faa8aadc0f83d19: start","job_status":"start","queue_duration_s":1.912876,"scheduling_latency_s":1.912894}2573gitlab # [ 437.401982] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.200Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5bf0519f4faa8aadc0f83d19","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5bf0519f4faa8aadc0f83d19: done: 0.025938 sec","job_status":"done","queue_duration_s":1.912876,"scheduling_latency_s":1.912894,"redis_calls":13,"redis_duration_s":0.00425,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001313,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002937,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017801,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025938,"completed_at":"2026-09-13T10:17:58.200Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2574gitlab # [ 437.441000] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.203Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb27e4eebaba14c4427a056f","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb27e4eebaba14c4427a056f: start","job_status":"start","queue_duration_s":1.941919,"scheduling_latency_s":1.941966}2575gitlab # [ 437.453229] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.240Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb27e4eebaba14c4427a056f","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb27e4eebaba14c4427a056f: done: 0.037366 sec","job_status":"done","queue_duration_s":1.941919,"scheduling_latency_s":1.941966,"redis_calls":13,"redis_duration_s":0.004542,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001519,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003023,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024568,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.037366,"completed_at":"2026-09-13T10:17:58.240Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2576gitlab # [ 437.491712] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.244Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"afc240e0a5d68617d95775f7","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-afc240e0a5d68617d95775f7: start","job_status":"start","queue_duration_s":1.9835,"scheduling_latency_s":1.983529}2577gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/456921b37007e803dac208d4de47f6e13ba8b9e4/statuses2578gitlab # [ 437.504133] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.275Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"afc240e0a5d68617d95775f7","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-afc240e0a5d68617d95775f7: done: 0.030811 sec","job_status":"done","queue_duration_s":1.9835,"scheduling_latency_s":1.983529,"redis_calls":13,"redis_duration_s":0.004575,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001455,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00312,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021464,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.030811,"completed_at":"2026-09-13T10:17:58.275Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2579gitlab # [ 437.541526] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.277Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"39fe8967e7b04e3ae4fa33c6","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-39fe8967e7b04e3ae4fa33c6: start","job_status":"start","queue_duration_s":2.016546,"scheduling_latency_s":2.016577}2580gitlab # [ 437.552789] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.307Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"39fe8967e7b04e3ae4fa33c6","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-39fe8967e7b04e3ae4fa33c6: done: 0.029899 sec","job_status":"done","queue_duration_s":2.016546,"scheduling_latency_s":2.016577,"redis_calls":13,"redis_duration_s":0.004891,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001469,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003422,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021087,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.029899,"completed_at":"2026-09-13T10:17:58.307Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2581gitlab # [ 437.590333] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.309Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16ebafc3c9bb4f6aaa924a48","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16ebafc3c9bb4f6aaa924a48: start","job_status":"start","queue_duration_s":2.048438,"scheduling_latency_s":2.048462}2582gitlab # [ 437.602644] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.332Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16ebafc3c9bb4f6aaa924a48","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16ebafc3c9bb4f6aaa924a48: done: 0.022733 sec","job_status":"done","queue_duration_s":2.048438,"scheduling_latency_s":2.048462,"redis_calls":13,"redis_duration_s":0.003494,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001137,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002357,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014881,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022733,"completed_at":"2026-09-13T10:17:58.332Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2583gitlab # [ 437.640909] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.334Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fce25ca6be20623e9c1de68b","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fce25ca6be20623e9c1de68b: start","job_status":"start","queue_duration_s":2.072919,"scheduling_latency_s":2.072939}2584gitlab # [ 437.653531] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.353Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fce25ca6be20623e9c1de68b","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fce25ca6be20623e9c1de68b: done: 0.019659 sec","job_status":"done","queue_duration_s":2.072919,"scheduling_latency_s":2.072939,"redis_calls":13,"redis_duration_s":0.003104,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001067,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002037,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012709,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019659,"completed_at":"2026-09-13T10:17:58.353Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2585gitlab # [ 437.691912] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.355Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dfcb756e12eeb1631648eb29","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dfcb756e12eeb1631648eb29: start","job_status":"start","queue_duration_s":2.094186,"scheduling_latency_s":2.094206}2586gitlab # [ 437.704139] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.376Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dfcb756e12eeb1631648eb29","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dfcb756e12eeb1631648eb29: done: 0.020538 sec","job_status":"done","queue_duration_s":2.094186,"scheduling_latency_s":2.094206,"redis_calls":13,"redis_duration_s":0.003841,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001433,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002408,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013094,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020538,"completed_at":"2026-09-13T10:17:58.376Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2587gitlab # [ 437.742242] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.377Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e829d32e904deb11d5ac764c","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e829d32e904deb11d5ac764c: start","job_status":"start","queue_duration_s":2.116363,"scheduling_latency_s":2.116384}2588gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/456921b37007e803dac208d4de47f6e13ba8b9e4/statuses, in 0.26 seconds)2589gitlab # [ 437.754690] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.396Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e829d32e904deb11d5ac764c","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e829d32e904deb11d5ac764c: done: 0.018783 sec","job_status":"done","queue_duration_s":2.116363,"scheduling_latency_s":2.116384,"redis_calls":13,"redis_duration_s":0.0031780000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001046,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002132,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011877,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018783,"completed_at":"2026-09-13T10:17:58.396Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2590gitlab # [ 437.793580] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.398Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1b502c78fc4e08a431625a31","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1b502c78fc4e08a431625a31: start","job_status":"start","queue_duration_s":2.136829,"scheduling_latency_s":2.136849}2591gitlab # [ 437.805823] gitaly[966]: time="2026-09-13T10:17:59.636Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4C5FDDXCTH47E5GE38B5G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:18:29.634" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:17:59.635" grpc.time_ms=1.287 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2592gitlab # [ 437.818770] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:17:59 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/456921b37007e803dac208d4de47f6e13ba8b9e4/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1572593gitlab # [ 437.821391] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.421Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1b502c78fc4e08a431625a31","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1b502c78fc4e08a431625a31: done: 0.022948 sec","job_status":"done","queue_duration_s":2.136829,"scheduling_latency_s":2.136849,"redis_calls":13,"redis_duration_s":0.006285000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004133,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002152,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012964,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022948,"completed_at":"2026-09-13T10:17:58.421Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2594gitlab # [ 437.860079] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.422Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc930564907f3ec0f6e80d57","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc930564907f3ec0f6e80d57: start","job_status":"start","queue_duration_s":2.161467,"scheduling_latency_s":2.161488}2595gitlab # [ 437.872143] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.447Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc930564907f3ec0f6e80d57","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc930564907f3ec0f6e80d57: done: 0.025106 sec","job_status":"done","queue_duration_s":2.161467,"scheduling_latency_s":2.161488,"redis_calls":13,"redis_duration_s":0.006881999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002343,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014005,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025106,"completed_at":"2026-09-13T10:17:58.447Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2596gitlab # [ 437.911207] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.449Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f5c2972ed14ea405089f707d","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f5c2972ed14ea405089f707d: start","job_status":"start","queue_duration_s":2.188436,"scheduling_latency_s":2.18846}2597gitlab # [ 437.923709] nix-daemon[3797]: accepted connection from pid 4876, user nixbot2598gitlab # [ 437.924683] nix-daemon[3797]: accepted connection from pid 4838, user nixbot2599gitlab # [ 437.925617] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.465Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f5c2972ed14ea405089f707d","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f5c2972ed14ea405089f707d: done: 0.016197 sec","job_status":"done","queue_duration_s":2.188436,"scheduling_latency_s":2.18846,"redis_calls":13,"redis_duration_s":0.002888,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000969,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001919,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009893,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016197,"completed_at":"2026-09-13T10:17:58.465Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2600gitlab # [ 437.965326] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.467Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fcf6395e15d14bcc5aa20575","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fcf6395e15d14bcc5aa20575: start","job_status":"start","queue_duration_s":2.206029,"scheduling_latency_s":2.206046}2601gitlab # [ 437.978000] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.483Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fcf6395e15d14bcc5aa20575","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fcf6395e15d14bcc5aa20575: done: 0.016349 sec","job_status":"done","queue_duration_s":2.206029,"scheduling_latency_s":2.206046,"redis_calls":13,"redis_duration_s":0.00303,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001032,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001998,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010351,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016349,"completed_at":"2026-09-13T10:17:58.483Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2602gitlab # [ 438.017185] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.485Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae376886e99030c3a95c497a","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae376886e99030c3a95c497a: start","job_status":"start","queue_duration_s":2.223845,"scheduling_latency_s":2.223863}2603gitlab # [ 438.029333] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.500Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae376886e99030c3a95c497a","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae376886e99030c3a95c497a: done: 0.015413 sec","job_status":"done","queue_duration_s":2.223845,"scheduling_latency_s":2.223863,"redis_calls":13,"redis_duration_s":0.0030759999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001384,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001692,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009429,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015413,"completed_at":"2026-09-13T10:17:58.500Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2604gitlab # [ 438.067696] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.501Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"62278a523a3375be0f6e6b14","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-62278a523a3375be0f6e6b14: start","job_status":"start","queue_duration_s":2.240573,"scheduling_latency_s":2.240589}2605gitlab # [ 438.080033] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.528Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"62278a523a3375be0f6e6b14","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-62278a523a3375be0f6e6b14: done: 0.026875 sec","job_status":"done","queue_duration_s":2.240573,"scheduling_latency_s":2.240589,"redis_calls":13,"redis_duration_s":0.0047539999999999995,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001625,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003129,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018601,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026875,"completed_at":"2026-09-13T10:17:58.528Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2606gitlab # [ 438.118226] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.531Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"47afb8dad4d44c283fc54a68","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-47afb8dad4d44c283fc54a68: start","job_status":"start","queue_duration_s":2.270453,"scheduling_latency_s":2.270493}2607gitlab # [ 438.130458] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.732Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"47afb8dad4d44c283fc54a68","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-47afb8dad4d44c283fc54a68: done: 0.200778 sec","job_status":"done","queue_duration_s":2.270453,"scheduling_latency_s":2.270493,"redis_calls":13,"redis_duration_s":0.00397,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001366,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002604,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.189599,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.200778,"completed_at":"2026-09-13T10:17:58.732Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2608gitlab # [ 438.168489] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.734Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cd306d77a537a91bfd1cf421","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cd306d77a537a91bfd1cf421: start","job_status":"start","queue_duration_s":2.47313,"scheduling_latency_s":2.47315}2609gitlab # [ 438.180311] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.751Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cd306d77a537a91bfd1cf421","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cd306d77a537a91bfd1cf421: done: 0.016552 sec","job_status":"done","queue_duration_s":2.47313,"scheduling_latency_s":2.47315,"redis_calls":13,"redis_duration_s":0.0034449999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000963,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002482,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010364,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016552,"completed_at":"2026-09-13T10:17:58.750Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2610gitlab # [ 438.218425] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.752Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"318713481055f8f3622584bd","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-318713481055f8f3622584bd: start","job_status":"start","queue_duration_s":2.491074,"scheduling_latency_s":2.49109}2611gitlab # [ 438.231623] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.768Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"318713481055f8f3622584bd","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-318713481055f8f3622584bd: done: 0.016153 sec","job_status":"done","queue_duration_s":2.491074,"scheduling_latency_s":2.49109,"redis_calls":13,"redis_duration_s":0.002827,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000973,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001854,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00962,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016153,"completed_at":"2026-09-13T10:17:58.768Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2612gitlab # [ 438.269971] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.770Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4952d609b42fe36b1618e2b2","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4952d609b42fe36b1618e2b2: start","job_status":"start","queue_duration_s":2.50951,"scheduling_latency_s":2.509528}2613gitlab # [ 438.282143] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.787Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4952d609b42fe36b1618e2b2","created_at":"2026-09-13T10:17:56.128Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-357b8a2d549c4656","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8dbda4c93a70ac9183560730da4d565f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.261Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4952d609b42fe36b1618e2b2: done: 0.016152 sec","job_status":"done","queue_duration_s":2.50951,"scheduling_latency_s":2.509528,"redis_calls":13,"redis_duration_s":0.002825,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000996,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001829,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01034,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016152,"completed_at":"2026-09-13T10:17:58.786Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2614gitlab # [ 438.320043] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.788Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"749704a8f08140e64becb44f","created_at":"2026-09-13T10:17:56.292Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-0ab19408396640c1","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3200b66ac7573fb76a7a6891c63fcc95","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.304Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-749704a8f08140e64becb44f: start","job_status":"start","queue_duration_s":2.484305,"scheduling_latency_s":2.484321}2615gitlab # [ 438.332034] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.802Z","message":"No existing merge request to be cleaned up."}2616gitlab # [ 438.333693] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.811Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"749704a8f08140e64becb44f","created_at":"2026-09-13T10:17:56.292Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-0ab19408396640c1","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3200b66ac7573fb76a7a6891c63fcc95","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.304Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-749704a8f08140e64becb44f: done: 0.022989 sec","job_status":"done","queue_duration_s":2.484305,"scheduling_latency_s":2.484321,"redis_calls":10,"redis_duration_s":0.0025859999999999998,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0011,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001486,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01474,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022989,"completed_at":"2026-09-13T10:17:58.811Z","load_balancing_strategy":"primary","db_duration_s":0.003569,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2617gitlab # [ 438.371479] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.812Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"13716caffdfbaa8116d24e1d","created_at":"2026-09-13T10:17:56.292Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-0ab19408396640c1","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3200b66ac7573fb76a7a6891c63fcc95","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.304Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-13716caffdfbaa8116d24e1d: start","job_status":"start","queue_duration_s":2.50876,"scheduling_latency_s":2.508777}2618gitlab # [ 438.383451] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.825Z","message":"No existing merge request to be cleaned up."}2619gitlab # [ 438.385183] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.835Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"13716caffdfbaa8116d24e1d","created_at":"2026-09-13T10:17:56.292Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-0ab19408396640c1","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3200b66ac7573fb76a7a6891c63fcc95","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.304Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-13716caffdfbaa8116d24e1d: done: 0.022956 sec","job_status":"done","queue_duration_s":2.50876,"scheduling_latency_s":2.508777,"redis_calls":10,"redis_duration_s":0.0026550000000000002,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001111,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001544,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01417,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022956,"completed_at":"2026-09-13T10:17:58.835Z","load_balancing_strategy":"primary","db_duration_s":0.004027,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2620gitlab # [ 438.423422] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.837Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"597641163c14f2cefdf5d328","created_at":"2026-09-13T10:17:56.292Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-0ab19408396640c1","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3200b66ac7573fb76a7a6891c63fcc95","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.304Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-597641163c14f2cefdf5d328: start","job_status":"start","queue_duration_s":2.533155,"scheduling_latency_s":2.533172}2621gitlab # [ 438.436671] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.849Z","message":"No existing merge request to be cleaned up."}2622gitlab # [ 438.438342] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.859Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"597641163c14f2cefdf5d328","created_at":"2026-09-13T10:17:56.292Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-0ab19408396640c1","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3200b66ac7573fb76a7a6891c63fcc95","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.304Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-597641163c14f2cefdf5d328: done: 0.022015 sec","job_status":"done","queue_duration_s":2.533155,"scheduling_latency_s":2.533172,"redis_calls":10,"redis_duration_s":0.0026449999999999998,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001052,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001593,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013784,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022015,"completed_at":"2026-09-13T10:17:58.859Z","load_balancing_strategy":"primary","db_duration_s":0.00257,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2623gitlab # [ 438.475932] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.860Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d879c73951e52677d3d031b6","created_at":"2026-09-13T10:17:56.292Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-0ab19408396640c1","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3200b66ac7573fb76a7a6891c63fcc95","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.304Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d879c73951e52677d3d031b6: start","job_status":"start","queue_duration_s":2.556679,"scheduling_latency_s":2.556695}2624gitlab # [ 438.488233] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.873Z","message":"No existing merge request to be cleaned up."}2625gitlab # [ 438.489866] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:17:58.881Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d879c73951e52677d3d031b6","created_at":"2026-09-13T10:17:56.292Z","trace_propagation_headers":{"sentry-trace":"45f78ed2e8114ad2bf8b99d92118620c-0ab19408396640c1","baggage":"sentry-trace_id=45f78ed2e8114ad2bf8b99d92118620c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3200b66ac7573fb76a7a6891c63fcc95","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:17:56.304Z","job_size_bytes":2,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d879c73951e52677d3d031b6: done: 0.021059 sec","job_status":"done","queue_duration_s":2.556679,"scheduling_latency_s":2.556695,"redis_calls":10,"redis_duration_s":0.002827,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001289,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001538,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013598,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021059,"completed_at":"2026-09-13T10:17:58.881Z","load_balancing_strategy":"primary","db_duration_s":0.002542,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2626gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/456921b37007e803dac208d4de47f6e13ba8b9e4/statuses2627gitlab # [ 438.978071] gitaly[966]: time="2026-09-13T10:18:00.885Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4C6PGFP97NX40B6CENM2W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:18:30.883" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:18:00.884" grpc.time_ms=1.395 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2628gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/456921b37007e803dac208d4de47f6e13ba8b9e4/statuses, in 0.24 seconds)2629gitlab # [ 439.032669] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:18:00 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/456921b37007e803dac208d4de47f6e13ba8b9e4/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1552630gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/456921b37007e803dac208d4de47f6e13ba8b9e4/statuses2631gitlab # [ 440.190803] gitaly[966]: time="2026-09-13T10:18:02.098Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4C7WDZE6PT0685VBAB83J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:18:32.095" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:18:02.096" grpc.time_ms=1.483 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2632gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/456921b37007e803dac208d4de47f6e13ba8b9e4/statuses, in 0.21 seconds)2633gitlab # [ 440.247139] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:18:02 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/456921b37007e803dac208d4de47f6e13ba8b9e4/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1572634gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/456921b37007e803dac208d4de47f6e13ba8b9e4/statuses2635gitlab # [ 441.402621] gitaly[966]: time="2026-09-13T10:18:03.310Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4C929FDSMC7N8WYQKW9MV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:18:33.307" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:18:03.308" grpc.time_ms=1.379 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2636gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/456921b37007e803dac208d4de47f6e13ba8b9e4/statuses, in 0.21 seconds)2637gitlab # [ 441.457938] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:18:03 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/456921b37007e803dac208d4de47f6e13ba8b9e4/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1552638gitlab # [ 441.981538] nix-daemon[3797]: accepted connection from pid 4838, user nixbot2639gitlab # [ 442.156161] gitaly[966]: time="2026-09-13T10:18:04.065Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4C9S86VCBCCGEKAEZT51V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:18:34.062" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:18:04.063" grpc.time_ms=1.291 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2640gitlab # [ 442.252810] gitaly[966]: time="2026-09-13T10:18:04.161Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=10 component=gitaly.StreamServerInterceptor correlation_id=01M2D4C9S86VCBCCGEKAEZT51V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:18:14.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:18:04.143" grpc.time_ms=18.348 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2641gitlab # [ 442.335096] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:18:04.242Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"747a1b77f482f72bd2fff1e2","created_at":"2026-09-13T10:18:04.231Z","trace_propagation_headers":{"sentry-trace":"bb63855ee80848f3a1f8f96e6459ff4e-87b2149db8084070","baggage":"sentry-trace_id=bb63855ee80848f3a1f8f96e6459ff4e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D4C9S86VCBCCGEKAEZT51V","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-13T10:18:04.239Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-747a1b77f482f72bd2fff1e2: start","job_status":"start","queue_duration_s":0.002716,"scheduling_latency_s":0.002786}2642gitlab # [ 442.381371] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:18:04 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/456921b37007e803dac208d4de47f6e13ba8b9e4 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 3272643gitlab # [ 442.388252] nixbot[3628]: {"ts": "2026-09-13T10:18:04.275Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/456921b37007e803dac208d4de47f6e13ba8b9e4 \"HTTP/1.1 201 Created\""}2644gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/456921b37007e803dac208d4de47f6e13ba8b9e4/statuses2645gitlab # [ 442.473075] gitaly[966]: time="2026-09-13T10:18:04.381Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4CA3VC3CF0C53NBZPTQB5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:18:34.375" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:18:04.376" grpc.time_ms=3.47 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2646gitlab # [ 442.557290] gitaly[966]: time="2026-09-13T10:18:04.465Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=14 component=gitaly.StreamServerInterceptor correlation_id=01M2D4CA3VC3CF0C53NBZPTQB5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:18:14.207" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:18:04.448" grpc.time_ms=17.234 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2647gitlab # [ 442.590774] gitaly[966]: time="2026-09-13T10:18:04.499Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D4C9S86VCBCCGEKAEZT51V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:18:04.496" grpc.time_ms=2.304 pid=966 span.kind=server system=grpc2648gitlab # [ 442.607452] gitaly[966]: time="2026-09-13T10:18:04.509Z" level=info msg="finished unary call with code OK" catfile.duration_ms=5 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=5 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4CA85MDW3GYQ3EP0YX6KV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:18:34.501" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:18:04.503" grpc.time_ms=5.693 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2649gitlab # [ 442.625467] gitaly[966]: time="2026-09-13T10:18:04.533Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D4C9S86VCBCCGEKAEZT51V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:18:04.532" grpc.time_ms=0.632 pid=966 span.kind=server system=grpc2650gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/456921b37007e803dac208d4de47f6e13ba8b9e4/statuses, in 0.18 seconds)2651gitlab # [ 442.640804] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:18:04 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/456921b37007e803dac208d4de47f6e13ba8b9e4/statuses HTTP/1.1" 200 677 "" "curl/8.21.0" 1272652gitlab # [ 442.655284] gitaly[966]: time="2026-09-13T10:18:04.564Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D4C9S86VCBCCGEKAEZT51V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:18:04.563" grpc.time_ms=0.397 pid=966 span.kind=server system=grpc2653gitlab # [ 442.674519] gitaly[966]: time="2026-09-13T10:18:04.568Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M2D4C9S86VCBCCGEKAEZT51V forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T10:18:34.460" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:18:04.460" operations="[repack_refs]" pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2654gitlab # [ 442.691484] gitaly[966]: time="2026-09-13T10:18:04.571Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2D4C9S86VCBCCGEKAEZT51V diskcache=339f7dd4-a96d-4644-b0a7-9738d700074f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T10:18:34.460" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:18:04.460" pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2655gitlab # [ 442.706966] nixbot[3628]: {"ts": "2026-09-13T10:18:04.571Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/456921b37007e803dac208d4de47f6e13ba8b9e4 \"HTTP/1.1 201 Created\""}2656gitlab # [ 442.711152] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:18:04 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/456921b37007e803dac208d4de47f6e13ba8b9e4 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 2852657gitlab # [ 442.715850] gitaly[966]: time="2026-09-13T10:18:04.571Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=102 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7331 command.oublock=0 command.real_time_ms=105 command.spawn_token_fork_ms=1 command.system_time_ms=42 command.user_time_ms=60 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4C9S86VCBCCGEKAEZT51V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T10:18:34.460" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:18:04.460" grpc.time_ms=110.193 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2658gitlab # [ 442.734032] gitaly[966]: time="2026-09-13T10:18:04.572Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01M2D4C9S86VCBCCGEKAEZT51V grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T10:18:34.460" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:18:04.460" pid=966 remote_ip="2001:db8:1::1" repository_info="{false {6 702 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-09-13 10:16:09.31449214 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root2659gitlab # [ 442.748914] gitaly[966]: time="2026-09-13T10:18:04.602Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01M2D4C9S86VCBCCGEKAEZT51V grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T10:18:34.460" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:18:04.460" pid=966 remote_ip="2001:db8:1::1" span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root2660gitlab # [ 442.763875] gitaly[966]: time="2026-09-13T10:18:04.604Z" level=info msg="optimized repository" clean_stale_data_time=30.5437ms clean_work_trees_time=1.32558ms commit_graph_time=600ns correlation_id=01M2D4C9S86VCBCCGEKAEZT51V gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="1.46µs" pid=966 prune_time=520ns relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="2.6µs" storage_name=default system=housekeeping total_time=31.92872ms2661gitlab # [ 442.770597] gitaly[966]: time="2026-09-13T10:18:04.604Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M2D4C9S86VCBCCGEKAEZT51V grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T10:18:34.460" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:18:04.460" pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2662gitlab # [ 442.780646] gitaly[966]: time="2026-09-13T10:18:04.660Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4CADDCBXJWA3KMSXN45EC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:18:34.059" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:18:04.659" grpc.time_ms=0.544 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2663gitlab # [ 442.793285] gitaly[966]: time="2026-09-13T10:18:04.665Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D4C9S86VCBCCGEKAEZT51V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-13T10:18:34.663" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:18:04.663" grpc.time_ms=1.763 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2664gitlab # [ 442.805814] gitaly[966]: time="2026-09-13T10:18:04.709Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4C9S86VCBCCGEKAEZT51V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:18:34.707" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:18:04.708" grpc.time_ms=0.964 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2665gitlab # [ 442.818109] gitaly[966]: time="2026-09-13T10:18:04.721Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D4CADDCBXJWA3KMSXN45EC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:18:14.129" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:18:04.709" grpc.time_ms=11.336 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2666gitlab # [ 442.831386] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:18:04.721Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"fad3f989c2b70040672675bf","created_at":"2026-09-13T10:18:04.720Z","trace_propagation_headers":{"sentry-trace":"bb63855ee80848f3a1f8f96e6459ff4e-fe5ef0a6cd434808","baggage":"sentry-trace_id=bb63855ee80848f3a1f8f96e6459ff4e,sentry-release=Unknown"},"correlation_id":"01M2D4C9S86VCBCCGEKAEZT51V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","duplicate-of":"747a1b77f482f72bd2fff1e2","job_size_bytes":3,"pid":3504,"job_status":"deduplicated","message":"PipelineProcessWorker JID-fad3f989c2b70040672675bf: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}2667gitlab # [ 442.846635] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:18:04.723Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"747a1b77f482f72bd2fff1e2","created_at":"2026-09-13T10:18:04.231Z","trace_propagation_headers":{"sentry-trace":"bb63855ee80848f3a1f8f96e6459ff4e-87b2149db8084070","baggage":"sentry-trace_id=bb63855ee80848f3a1f8f96e6459ff4e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D4C9S86VCBCCGEKAEZT51V","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-13T10:18:04.239Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":3504,"job_status":"rescheduled","message":"PipelineProcessWorker JID-747a1b77f482f72bd2fff1e2: rescheduled"}2668gitlab # [ 442.861590] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:18:04.730Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"747a1b77f482f72bd2fff1e2","created_at":"2026-09-13T10:18:04.231Z","trace_propagation_headers":{"sentry-trace":"bb63855ee80848f3a1f8f96e6459ff4e-87b2149db8084070","baggage":"sentry-trace_id=bb63855ee80848f3a1f8f96e6459ff4e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D4C9S86VCBCCGEKAEZT51V","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-13T10:18:04.239Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-747a1b77f482f72bd2fff1e2: done: 0.488421 sec","job_status":"done","queue_duration_s":0.002716,"scheduling_latency_s":0.002786,"gitaly_calls":3,"gitaly_duration_s":0.124531,"redis_calls":80,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.048234,"redis_read_bytes":240,"redis_write_bytes":21868,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.002093,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":11,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001535,"redis_cache_read_bytes":20,"redis_cache_write_bytes":948,"redis_queues_calls":25,"redis_queues_duration_s":0.004292,"redis_queues_read_bytes":28,"redis_queues_write_bytes":14468,"redis_queues_metadata_calls":23,"redis_queues_metadata_duration_s":0.007812,"redis_queues_metadata_read_bytes":112,"redis_queues_metadata_write_bytes":3754,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.024049,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00042,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.00594,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.06,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":16,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.013,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.017,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.029,"db_ci_duration_s":0.031,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.265237,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.4043224,"duration_s":0.488421,"completed_at":"2026-09-13T10:18:04.730Z","load_balancing_strategy":"primary","db_duration_s":0.063456,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2669gitlab # [ 442.912202] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:18:04.732Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"703f79d6a6d6354fdcb1dee0","created_at":"2026-09-13T10:18:04.440Z","trace_propagation_headers":{"sentry-trace":"bb63855ee80848f3a1f8f96e6459ff4e-fe5ef0a6cd434808","baggage":"sentry-trace_id=bb63855ee80848f3a1f8f96e6459ff4e,sentry-release=Unknown"},"correlation_id":"01M2D4C9S86VCBCCGEKAEZT51V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-09-13T10:18:04.446Z","job_size_bytes":23,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-703f79d6a6d6354fdcb1dee0: start","job_status":"start","queue_duration_s":0.285743,"scheduling_latency_s":0.285766}2670gitlab # [ 442.929703] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:18:04.787Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"703f79d6a6d6354fdcb1dee0","created_at":"2026-09-13T10:18:04.440Z","trace_propagation_headers":{"sentry-trace":"bb63855ee80848f3a1f8f96e6459ff4e-fe5ef0a6cd434808","baggage":"sentry-trace_id=bb63855ee80848f3a1f8f96e6459ff4e,sentry-release=Unknown"},"correlation_id":"01M2D4C9S86VCBCCGEKAEZT51V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-09-13T10:18:04.446Z","job_size_bytes":23,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-703f79d6a6d6354fdcb1dee0: done: 0.055367 sec","job_status":"done","queue_duration_s":0.285743,"scheduling_latency_s":0.285766,"redis_calls":7,"redis_duration_s":0.0015209999999999998,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001169,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000352,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03686,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.055367,"completed_at":"2026-09-13T10:18:04.787Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015545,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2671gitlab # [ 442.972354] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:18:04.788Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"fae4ac1fee19489b506c7d44","created_at":"2026-09-13T10:18:04.581Z","trace_propagation_headers":{"sentry-trace":"bb63855ee80848f3a1f8f96e6459ff4e-fe5ef0a6cd434808","baggage":"sentry-trace_id=bb63855ee80848f3a1f8f96e6459ff4e,sentry-release=Unknown"},"correlation_id":"01M2D4C9S86VCBCCGEKAEZT51V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-09-13T10:18:04.585Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-fae4ac1fee19489b506c7d44: start","job_status":"start","queue_duration_s":0.203267,"scheduling_latency_s":0.203287}2672gitlab # [ 442.988706] nixbot[3628]: {"ts": "2026-09-13T10:18:04.809Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/456921b37007e803dac208d4de47f6e13ba8b9e4 \"HTTP/1.1 201 Created\""}2673gitlab # [ 442.991337] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:18:04 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/456921b37007e803dac208d4de47f6e13ba8b9e4 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 2192674gitlab # [ 442.993889] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:18:04.822Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"fae4ac1fee19489b506c7d44","created_at":"2026-09-13T10:18:04.581Z","trace_propagation_headers":{"sentry-trace":"bb63855ee80848f3a1f8f96e6459ff4e-fe5ef0a6cd434808","baggage":"sentry-trace_id=bb63855ee80848f3a1f8f96e6459ff4e,sentry-release=Unknown"},"correlation_id":"01M2D4C9S86VCBCCGEKAEZT51V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-09-13T10:18:04.585Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-fae4ac1fee19489b506c7d44: done: 0.033611 sec","job_status":"done","queue_duration_s":0.203267,"scheduling_latency_s":0.203287,"redis_calls":7,"redis_duration_s":0.001706,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001162,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000544,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021943,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033611,"completed_at":"2026-09-13T10:18:04.822Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006643,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2675gitlab # [ 443.036189] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:18:04.830Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"2f35e3da5ac21f8b60170440","created_at":"2026-09-13T10:18:04.602Z","trace_propagation_headers":{"sentry-trace":"bb63855ee80848f3a1f8f96e6459ff4e-fe5ef0a6cd434808","baggage":"sentry-trace_id=bb63855ee80848f3a1f8f96e6459ff4e,sentry-release=Unknown"},"correlation_id":"01M2D4C9S86VCBCCGEKAEZT51V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-09-13T10:18:04.605Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-2f35e3da5ac21f8b60170440: start","job_status":"start","queue_duration_s":0.224756,"scheduling_latency_s":0.22478}2676gitlab # [ 443.051772] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:18:04.875Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"2f35e3da5ac21f8b60170440","created_at":"2026-09-13T10:18:04.602Z","trace_propagation_headers":{"sentry-trace":"bb63855ee80848f3a1f8f96e6459ff4e-fe5ef0a6cd434808","baggage":"sentry-trace_id=bb63855ee80848f3a1f8f96e6459ff4e,sentry-release=Unknown"},"correlation_id":"01M2D4C9S86VCBCCGEKAEZT51V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-09-13T10:18:04.605Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-2f35e3da5ac21f8b60170440: done: 0.044949 sec","job_status":"done","queue_duration_s":0.224756,"scheduling_latency_s":0.22478,"redis_calls":7,"redis_duration_s":0.003142,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001701,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001441,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.014,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025068,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.044949,"completed_at":"2026-09-13T10:18:04.875Z","load_balancing_strategy":"primary","db_duration_s":0.014073,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2677gitlab # [ 443.093344] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:18:04.877Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"913c4ba936ccf5be8bb69268","created_at":"2026-09-13T10:18:04.617Z","trace_propagation_headers":{"sentry-trace":"bb63855ee80848f3a1f8f96e6459ff4e-fe5ef0a6cd434808","baggage":"sentry-trace_id=bb63855ee80848f3a1f8f96e6459ff4e,sentry-release=Unknown"},"correlation_id":"01M2D4C9S86VCBCCGEKAEZT51V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-09-13T10:18:04.619Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-913c4ba936ccf5be8bb69268: start","job_status":"start","queue_duration_s":0.257557,"scheduling_latency_s":0.257579}2678gitlab # [ 443.108391] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:18:04.918Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"913c4ba936ccf5be8bb69268","created_at":"2026-09-13T10:18:04.617Z","trace_propagation_headers":{"sentry-trace":"bb63855ee80848f3a1f8f96e6459ff4e-fe5ef0a6cd434808","baggage":"sentry-trace_id=bb63855ee80848f3a1f8f96e6459ff4e,sentry-release=Unknown"},"correlation_id":"01M2D4C9S86VCBCCGEKAEZT51V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-09-13T10:18:04.619Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-913c4ba936ccf5be8bb69268: done: 0.041487 sec","job_status":"done","queue_duration_s":0.257557,"scheduling_latency_s":0.257579,"redis_calls":10,"redis_duration_s":0.002737,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001341,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000861,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000535,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025758,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.041487,"completed_at":"2026-09-13T10:18:04.918Z","load_balancing_strategy":"primary","db_duration_s":0.009491,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2679gitlab # [ 443.152188] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:18:04.920Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"7fef33c82ad2e478905fc235","created_at":"2026-09-13T10:18:04.621Z","trace_propagation_headers":{"sentry-trace":"bb63855ee80848f3a1f8f96e6459ff4e-fe5ef0a6cd434808","baggage":"sentry-trace_id=bb63855ee80848f3a1f8f96e6459ff4e,sentry-release=Unknown"},"correlation_id":"01M2D4C9S86VCBCCGEKAEZT51V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-09-13T10:18:04.624Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-7fef33c82ad2e478905fc235: start","job_status":"start","queue_duration_s":0.296339,"scheduling_latency_s":0.296362}2680gitlab # [ 443.168249] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:18:04.968Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"7fef33c82ad2e478905fc235","created_at":"2026-09-13T10:18:04.621Z","trace_propagation_headers":{"sentry-trace":"bb63855ee80848f3a1f8f96e6459ff4e-fe5ef0a6cd434808","baggage":"sentry-trace_id=bb63855ee80848f3a1f8f96e6459ff4e,sentry-release=Unknown"},"correlation_id":"01M2D4C9S86VCBCCGEKAEZT51V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-09-13T10:18:04.624Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-7fef33c82ad2e478905fc235: done: 0.047571 sec","job_status":"done","queue_duration_s":0.296339,"scheduling_latency_s":0.296362,"redis_calls":7,"redis_duration_s":0.002696,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002068,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000628,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026277,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.047571,"completed_at":"2026-09-13T10:18:04.968Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014288,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2681gitlab # [ 443.210499] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:18:04.972Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"bc5e4b5729c5d2916b06ecd1","created_at":"2026-09-13T10:18:04.625Z","trace_propagation_headers":{"sentry-trace":"bb63855ee80848f3a1f8f96e6459ff4e-fe5ef0a6cd434808","baggage":"sentry-trace_id=bb63855ee80848f3a1f8f96e6459ff4e,sentry-release=Unknown"},"correlation_id":"01M2D4C9S86VCBCCGEKAEZT51V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-09-13T10:18:04.629Z","job_size_bytes":19,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-bc5e4b5729c5d2916b06ecd1: start","job_status":"start","queue_duration_s":0.343412,"scheduling_latency_s":0.343461}2682gitlab # [ 443.226635] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:18:05.030Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"bc5e4b5729c5d2916b06ecd1","created_at":"2026-09-13T10:18:04.625Z","trace_propagation_headers":{"sentry-trace":"bb63855ee80848f3a1f8f96e6459ff4e-fe5ef0a6cd434808","baggage":"sentry-trace_id=bb63855ee80848f3a1f8f96e6459ff4e,sentry-release=Unknown"},"correlation_id":"01M2D4C9S86VCBCCGEKAEZT51V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-09-13T10:18:04.629Z","job_size_bytes":19,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-bc5e4b5729c5d2916b06ecd1: done: 0.058297 sec","job_status":"done","queue_duration_s":0.343412,"scheduling_latency_s":0.343461,"redis_calls":8,"redis_duration_s":0.002695,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002147,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000548,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.042149,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.058297,"completed_at":"2026-09-13T10:18:05.030Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005367,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2683gitlab # [ 443.269408] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:18:05.034Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"71c3412fab060a2ee5b73c33","created_at":"2026-09-13T10:18:04.630Z","trace_propagation_headers":{"sentry-trace":"bb63855ee80848f3a1f8f96e6459ff4e-fe5ef0a6cd434808","baggage":"sentry-trace_id=bb63855ee80848f3a1f8f96e6459ff4e,sentry-release=Unknown"},"correlation_id":"01M2D4C9S86VCBCCGEKAEZT51V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-09-13T10:18:04.633Z","job_size_bytes":66,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-71c3412fab060a2ee5b73c33: start","job_status":"start","queue_duration_s":0.401205,"scheduling_latency_s":0.401247}2684gitlab # [ 443.286749] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:18:05.088Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"71c3412fab060a2ee5b73c33","created_at":"2026-09-13T10:18:04.630Z","trace_propagation_headers":{"sentry-trace":"bb63855ee80848f3a1f8f96e6459ff4e-fe5ef0a6cd434808","baggage":"sentry-trace_id=bb63855ee80848f3a1f8f96e6459ff4e,sentry-release=Unknown"},"correlation_id":"01M2D4C9S86VCBCCGEKAEZT51V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-09-13T10:18:04.633Z","job_size_bytes":66,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-71c3412fab060a2ee5b73c33: done: 0.05455 sec","job_status":"done","queue_duration_s":0.401205,"scheduling_latency_s":0.401247,"redis_calls":7,"redis_duration_s":0.0063,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005786,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000514,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036403,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.05455,"completed_at":"2026-09-13T10:18:05.088Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006052,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2685gitlab # [ 443.330456] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:18:05.091Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"8cfa7f2dcdd9cf28a5d4738a","created_at":"2026-09-13T10:18:04.634Z","trace_propagation_headers":{"sentry-trace":"bb63855ee80848f3a1f8f96e6459ff4e-fe5ef0a6cd434808","baggage":"sentry-trace_id=bb63855ee80848f3a1f8f96e6459ff4e,sentry-release=Unknown"},"correlation_id":"01M2D4C9S86VCBCCGEKAEZT51V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-13T10:18:04.637Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-8cfa7f2dcdd9cf28a5d4738a: start","job_status":"start","queue_duration_s":0.454471,"scheduling_latency_s":0.454502}2686gitlab # [ 443.347258] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:18:05.159Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"8cfa7f2dcdd9cf28a5d4738a","created_at":"2026-09-13T10:18:04.634Z","trace_propagation_headers":{"sentry-trace":"bb63855ee80848f3a1f8f96e6459ff4e-fe5ef0a6cd434808","baggage":"sentry-trace_id=bb63855ee80848f3a1f8f96e6459ff4e,sentry-release=Unknown"},"correlation_id":"01M2D4C9S86VCBCCGEKAEZT51V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-13T10:18:04.637Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-8cfa7f2dcdd9cf28a5d4738a: done: 0.068006 sec","job_status":"done","queue_duration_s":0.454471,"scheduling_latency_s":0.454502,"redis_calls":7,"redis_duration_s":0.004057,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003532,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000525,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.048007,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.068006,"completed_at":"2026-09-13T10:18:05.159Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012316,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2687gitlab # [ 443.390775] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:18:05.161Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"4084d2fc68d0694ea4ec64ee","created_at":"2026-09-13T10:18:04.642Z","trace_propagation_headers":{"sentry-trace":"bb63855ee80848f3a1f8f96e6459ff4e-fe5ef0a6cd434808","baggage":"sentry-trace_id=bb63855ee80848f3a1f8f96e6459ff4e,sentry-release=Unknown"},"correlation_id":"01M2D4C9S86VCBCCGEKAEZT51V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-09-13T10:18:04.644Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-4084d2fc68d0694ea4ec64ee: start","job_status":"start","queue_duration_s":0.516918,"scheduling_latency_s":0.516944}2688gitlab # [ 443.407023] gitaly[966]: time="2026-09-13T10:18:05.284Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D4CAZT6AN26WKZGXGGBDCJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:18:05.265" grpc.time_ms=19.532 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc2689gitlab # [ 443.420384] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:18:05 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1072690gitlab # [ 443.422464] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:18:05.199Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"4084d2fc68d0694ea4ec64ee","created_at":"2026-09-13T10:18:04.642Z","trace_propagation_headers":{"sentry-trace":"bb63855ee80848f3a1f8f96e6459ff4e-fe5ef0a6cd434808","baggage":"sentry-trace_id=bb63855ee80848f3a1f8f96e6459ff4e,sentry-release=Unknown"},"correlation_id":"01M2D4C9S86VCBCCGEKAEZT51V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-09-13T10:18:04.644Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-4084d2fc68d0694ea4ec64ee: done: 0.038139 sec","job_status":"done","queue_duration_s":0.516918,"scheduling_latency_s":0.516944,"redis_calls":7,"redis_duration_s":0.001568,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001148,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00042,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026282,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.038139,"completed_at":"2026-09-13T10:18:05.199Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005545,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2691gitlab # [ 443.464211] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:18:05.201Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"454701c2c57fa60977b6e6e2","created_at":"2026-09-13T10:18:04.644Z","trace_propagation_headers":{"sentry-trace":"bb63855ee80848f3a1f8f96e6459ff4e-fe5ef0a6cd434808","baggage":"sentry-trace_id=bb63855ee80848f3a1f8f96e6459ff4e,sentry-release=Unknown"},"correlation_id":"01M2D4C9S86VCBCCGEKAEZT51V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-13T10:18:04.647Z","job_size_bytes":6,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-454701c2c57fa60977b6e6e2: start","job_status":"start","queue_duration_s":0.554035,"scheduling_latency_s":0.55406}2692gitlab # [ 443.479550] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:18:05.222Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"454701c2c57fa60977b6e6e2","created_at":"2026-09-13T10:18:04.644Z","trace_propagation_headers":{"sentry-trace":"bb63855ee80848f3a1f8f96e6459ff4e-fe5ef0a6cd434808","baggage":"sentry-trace_id=bb63855ee80848f3a1f8f96e6459ff4e,sentry-release=Unknown"},"correlation_id":"01M2D4C9S86VCBCCGEKAEZT51V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-13T10:18:04.647Z","job_size_bytes":6,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-454701c2c57fa60977b6e6e2: done: 0.02121 sec","job_status":"done","queue_duration_s":0.554035,"scheduling_latency_s":0.55406,"redis_calls":7,"redis_duration_s":0.003559,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003146,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000413,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012181,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02121,"completed_at":"2026-09-13T10:18:05.222Z","load_balancing_strategy":"primary","db_duration_s":0.000522,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2693gitlab # [ 443.520745] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:18:05.228Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"bdb14fb20b1a22535bf1e3cc","created_at":"2026-09-13T10:18:04.647Z","trace_propagation_headers":{"sentry-trace":"bb63855ee80848f3a1f8f96e6459ff4e-fe5ef0a6cd434808","baggage":"sentry-trace_id=bb63855ee80848f3a1f8f96e6459ff4e,sentry-release=Unknown"},"correlation_id":"01M2D4C9S86VCBCCGEKAEZT51V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-09-13T10:18:04.650Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-bdb14fb20b1a22535bf1e3cc: start","job_status":"start","queue_duration_s":0.578482,"scheduling_latency_s":0.578504}2694gitlab # [ 443.536980] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:18:05.263Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"bdb14fb20b1a22535bf1e3cc","created_at":"2026-09-13T10:18:04.647Z","trace_propagation_headers":{"sentry-trace":"bb63855ee80848f3a1f8f96e6459ff4e-fe5ef0a6cd434808","baggage":"sentry-trace_id=bb63855ee80848f3a1f8f96e6459ff4e,sentry-release=Unknown"},"correlation_id":"01M2D4C9S86VCBCCGEKAEZT51V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-09-13T10:18:04.650Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-bdb14fb20b1a22535bf1e3cc: done: 0.034832 sec","job_status":"done","queue_duration_s":0.578482,"scheduling_latency_s":0.578504,"redis_calls":7,"redis_duration_s":0.00204,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001547,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000493,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022326,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034832,"completed_at":"2026-09-13T10:18:05.263Z","load_balancing_strategy":"primary","db_duration_s":0.00745,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2695gitlab # [ 443.577608] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:18:05.266Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"faf48ae6d80401b24be20811","created_at":"2026-09-13T10:18:04.723Z","trace_propagation_headers":{"sentry-trace":"bb63855ee80848f3a1f8f96e6459ff4e-fe5ef0a6cd434808","baggage":"sentry-trace_id=bb63855ee80848f3a1f8f96e6459ff4e,sentry-release=Unknown"},"correlation_id":"01M2D4C9S86VCBCCGEKAEZT51V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-13T10:18:04.725Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-faf48ae6d80401b24be20811: start","job_status":"start","queue_duration_s":0.540588,"scheduling_latency_s":0.540611}2696gitlab # [ 443.593015] gitlab-workhorse[885]: time="2026-09-13T10:18:05Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672697gitlab # [ 443.594826] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:18:05 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1112698gitlab # [ 443.597401] gitaly[966]: time="2026-09-13T10:18:05.398Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2D4CB37FMQQMSHMYBZHN5H4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:18:05.384" pid=966 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2699gitlab # [ 443.607875] gitaly[966]: time="2026-09-13T10:18:05.398Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=3 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4CB37FMQQMSHMYBZHN5H4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:18:05.384" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc2700gitlab # [ 443.621328] gitaly[966]: time="2026-09-13T10:18:05.399Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=3 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4CB37FMQQMSHMYBZHN5H4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:18:05.384" grpc.time_ms=14.94 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc2701gitlab # [ 443.636165] gitaly[966]: time="2026-09-13T10:18:05.410Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D4C9S86VCBCCGEKAEZT51V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:18:05.409" grpc.time_ms=0.498 pid=966 span.kind=server system=grpc2702gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/456921b37007e803dac208d4de47f6e13ba8b9e4/statuses2703gitlab # [ 443.645504] gitaly[966]: time="2026-09-13T10:18:05.446Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D4C9S86VCBCCGEKAEZT51V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:18:05.445" grpc.time_ms=1.154 pid=966 span.kind=server system=grpc2704gitlab # [ 443.654239] gitaly[966]: time="2026-09-13T10:18:05.498Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D4C9S86VCBCCGEKAEZT51V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:18:05.496" grpc.time_ms=0.856 pid=966 span.kind=server system=grpc2705gitlab # [ 443.663061] gitaly[966]: time="2026-09-13T10:18:05.509Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2D4C9S86VCBCCGEKAEZT51V diskcache=477d76ee-d40c-41bc-942f-8ef77f0d20da grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T10:18:35.367" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:18:05.368" pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2706gitlab # [ 443.671966] gitaly[966]: time="2026-09-13T10:18:05.509Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=120 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7299 command.oublock=0 command.real_time_ms=134 command.spawn_token_fork_ms=1 command.system_time_ms=53 command.user_time_ms=67 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4C9S86VCBCCGEKAEZT51V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T10:18:35.367" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:18:05.368" grpc.time_ms=140.99 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2707gitlab # [ 443.683949] gitaly[966]: time="2026-09-13T10:18:05.578Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D4C9S86VCBCCGEKAEZT51V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-13T10:18:35.577" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:18:05.577" grpc.time_ms=0.823 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2708gitlab # [ 443.706369] gitaly[966]: time="2026-09-13T10:18:05.615Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D4C9S86VCBCCGEKAEZT51V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:18:35.614" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:18:05.614" grpc.time_ms=0.614 pid=966 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2709gitlab # [ 443.718851] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:18:05.625Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"faf48ae6d80401b24be20811","created_at":"2026-09-13T10:18:04.723Z","trace_propagation_headers":{"sentry-trace":"bb63855ee80848f3a1f8f96e6459ff4e-fe5ef0a6cd434808","baggage":"sentry-trace_id=bb63855ee80848f3a1f8f96e6459ff4e,sentry-release=Unknown"},"correlation_id":"01M2D4C9S86VCBCCGEKAEZT51V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-13T10:18:04.725Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":3504,"job_status":"rescheduled","message":"PipelineProcessWorker JID-faf48ae6d80401b24be20811: rescheduled"}2710gitlab # [ 443.733458] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:18:05.633Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"faf48ae6d80401b24be20811","created_at":"2026-09-13T10:18:04.723Z","trace_propagation_headers":{"sentry-trace":"bb63855ee80848f3a1f8f96e6459ff4e-fe5ef0a6cd434808","baggage":"sentry-trace_id=bb63855ee80848f3a1f8f96e6459ff4e,sentry-release=Unknown"},"correlation_id":"01M2D4C9S86VCBCCGEKAEZT51V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-13T10:18:04.725Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-faf48ae6d80401b24be20811: done: 0.36786 sec","job_status":"done","queue_duration_s":0.540588,"scheduling_latency_s":0.540611,"gitaly_calls":3,"gitaly_duration_s":0.150848,"redis_calls":68,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.012816000000000001,"redis_read_bytes":149,"redis_write_bytes":18173,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000795,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":11,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001019,"redis_cache_read_bytes":20,"redis_cache_write_bytes":948,"redis_queues_calls":22,"redis_queues_duration_s":0.003905,"redis_queues_read_bytes":24,"redis_queues_write_bytes":12315,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.004145,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000388,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000408,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001361,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.04,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.015,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.015,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.022,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.165522,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.31742384,"duration_s":0.36786,"completed_at":"2026-09-13T10:18:05.633Z","load_balancing_strategy":"primary","db_duration_s":0.046179,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2711gitlab # [ 443.784742] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:18:05.640Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"45a0975f30de0dd77a530629","created_at":"2026-09-13T10:18:05.356Z","trace_propagation_headers":{"sentry-trace":"bb63855ee80848f3a1f8f96e6459ff4e-d0a64cafcfc4471b","baggage":"sentry-trace_id=bb63855ee80848f3a1f8f96e6459ff4e,sentry-release=Unknown"},"correlation_id":"01M2D4C9S86VCBCCGEKAEZT51V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-09-13T10:18:05.359Z","job_size_bytes":23,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-45a0975f30de0dd77a530629: start","job_status":"start","queue_duration_s":0.280989,"scheduling_latency_s":0.28102}2712gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/456921b37007e803dac208d4de47f6e13ba8b9e4/statuses, in 0.20 seconds)2713gitlab # [ 443.800974] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:18:05.690Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"45a0975f30de0dd77a530629","created_at":"2026-09-13T10:18:05.356Z","trace_propagation_headers":{"sentry-trace":"bb63855ee80848f3a1f8f96e6459ff4e-d0a64cafcfc4471b","baggage":"sentry-trace_id=bb63855ee80848f3a1f8f96e6459ff4e,sentry-release=Unknown"},"correlation_id":"01M2D4C9S86VCBCCGEKAEZT51V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-09-13T10:18:05.359Z","job_size_bytes":23,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-45a0975f30de0dd77a530629: done: 0.049806 sec","job_status":"done","queue_duration_s":0.280989,"scheduling_latency_s":0.28102,"redis_calls":7,"redis_duration_s":0.001353,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000945,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000408,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03539,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.049806,"completed_at":"2026-09-13T10:18:05.690Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008659,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2714(finished: subtest: eval and build post commit statuses, in 11.05 seconds)2715(finished: run the VM test script, in 513.90 seconds)2716gitlab # [ 443.843392] sidekiq-cluster[3504]: {"severity":"INFO","time":"2026-09-13T10:18:05.691Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"1fef1b6a1fbe333ac218485b","created_at":"2026-09-13T10:18:05.517Z","trace_propagation_headers":{"sentry-trace":"bb63855ee80848f3a1f8f96e6459ff4e-d0a64cafcfc4471b","baggage":"sentry-trace_id=bb63855ee80848f3a1f8f96e6459ff4e,sentry-release=Unknown"},"correlation_id":"01M2D4C9S86VCBCCGEKAEZT51V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-09-13T10:18:05.521Z","job_size_bytes":3,"pid":3504,"sidekiq_tid":"m8rc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-1fef1b6a1fbe333ac218485b: start","job_status":"start","queue_duration_s":0.170769,"scheduling_latency_s":0.17079}2717gitlab # [ 443.859781] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:18:05 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 2052718gitlab # [ 443.863061] gitlab-workhorse[885]: time="2026-09-13T10:18:05Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672719gitlab # [ 443.866474] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:18:05 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 852720gitlab # [ 443.868431] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/09/13:10:18:05 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/456921b37007e803dac208d4de47f6e13ba8b9e4/statuses HTTP/1.1" 200 1334 "" "curl/8.21.0" 1092721test script finished in 513.92s2722cleanup2723kill QemuMachine (pid 46)2724gitlab # [ 443.872043] gitaly[966]: time="2026-09-13T10:18:05.645Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.usqemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/41m77i1296n33p6liin8ynr6wh3h6b7m-python3-3.14.6/bin/python3.14)2725(finished: cleanup, in 1.60 seconds)2726additionally exposed symbols:2727 gitlab,2728 vlan1,2729 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh2730[]2731[{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-09-13T10:16:22.330Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]2732[]2733[]2734[]2735[]2736[]2737[]2738[]2739[]2740[]2741[]2742[]2743[]2744[{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '456921b37007e803dac208d4de47f6e13ba8b9e4', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'evaluating', 'eval_warnings': None, 'error': None, 'created_at': '2026-09-13T10:17:54.260597Z', 'started_at': '2026-09-13T10:17:54.624820Z', 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': None}]2745[{'id': 5, 'sha': '456921b37007e803dac208d4de47f6e13ba8b9e4', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-13T10:17:54.593Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2746[{'id': 5, 'sha': '456921b37007e803dac208d4de47f6e13ba8b9e4', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-13T10:17:54.593Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2747[{'id': 5, 'sha': '456921b37007e803dac208d4de47f6e13ba8b9e4', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-13T10:17:54.593Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2748[{'id': 5, 'sha': '456921b37007e803dac208d4de47f6e13ba8b9e4', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-13T10:17:54.593Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2749[{'id': 5, 'sha': '456921b37007e803dac208d4de47f6e13ba8b9e4', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-13T10:17:54.593Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2750[{'id': 5, 'sha': '456921b37007e803dac208d4de47f6e13ba8b9e4', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-13T10:17:54.593Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2751[{'id': 5, 'sha': '456921b37007e803dac208d4de47f6e13ba8b9e4', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-13T10:17:54.593Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2752[{'id': 5, 'sha': '456921b37007e803dac208d4de47f6e13ba8b9e4', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-13T10:17:54.593Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2753[{'id': 5, 'sha': '456921b37007e803dac208d4de47f6e13ba8b9e4', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 9s (1 warning)', 'created_at': '2026-09-13T10:17:54.593Z', 'started_at': None, 'finished_at': '2026-09-13T10:18:04.209Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2754[{'id': 5, 'sha': '456921b37007e803dac208d4de47f6e13ba8b9e4', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 9s (1 warning)', 'created_at': '2026-09-13T10:17:54.593Z', 'started_at': None, 'finished_at': '2026-09-13T10:18:04.209Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': '456921b37007e803dac208d4de47f6e13ba8b9e4', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '0 attributes built', 'created_at': '2026-09-13T10:18:04.535Z', 'started_at': None, 'finished_at': '2026-09-13T10:18:04.759Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]