vm-test-run-nixbot-gitlab
checks.aarch64-linux.nixbot-gitlab
· build #500
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 3600 seconds6run the VM test script7start all VMs8gitlab: starting vm9gitlab # Disk image does not exist, creating the virtualisation disk image...10gitlab: QEMU running (pid 46)11gitlab # Formatting '/build/vm-state-gitlab/tmp.JX8PSkVwVt', fmt=raw size=107374182412gitlab # mke2fs 1.47.4 (6-Mar-2025)13gitlab # Discarding device blocks: 0/262144 done14gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes15gitlab # Filesystem UUID: f0fd218e-56af-473f-84ce-fdcfeaf8fa9316(finished: start all VMs, in 69.51 seconds)17gitlab # Superblock backups stored on blocks:18gitlab: waiting for unit setup-gitlab.service19gitlab # 32768, 98304, 163840, 22937620gitlab: waiting for the VM to finish booting21gitlab # 22gitlab # Allocating group tables: 0/8 done23gitlab # Writing inode tables: 0/8 done24gitlab # Creating journal (8192 blocks): done25gitlab # Writing superblocks and filesystem accounting information: 0/8 done26gitlab # 27gitlab # Virtualisation disk image created.28gitlab # Creating Nix store image...29gitlab # Created Nix store image.30gitlab # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]31gitlab # [ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Thu Aug 6 17:27:09 UTC 202632gitlab # [ 0.000000] KASLR enabled33gitlab # [ 0.000000] random: crng init done34gitlab # [ 0.000000] Machine model: linux,dummy-virt35gitlab # [ 0.000000] efi: UEFI not found.36gitlab # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT37gitlab # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x00000001bfffffff]38gitlab # [ 0.000000] NODE_DATA(0) allocated [mem 0x1bfdeb740-0x1bfdeeebf]39gitlab # [ 0.000000] Zone ranges:40gitlab # [ 0.000000] DMA [mem 0x0000000040000000-0x00000000ffffffff]41gitlab # [ 0.000000] DMA32 empty42gitlab # [ 0.000000] Normal [mem 0x0000000100000000-0x00000001bfffffff]43gitlab # [ 0.000000] Device empty44gitlab # [ 0.000000] Movable zone start for each node45gitlab # [ 0.000000] Early memory node ranges46gitlab # [ 0.000000] node 0: [mem 0x0000000040000000-0x00000001bfffffff]47gitlab # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000001bfffffff]48gitlab # [ 0.000000] cma: Reserved 32 MiB at 0x00000000fe00000049gitlab # [ 0.000000] psci: probing for conduit method from DT.50gitlab # [ 0.000000] psci: PSCIv1.3 detected in firmware.51gitlab # [ 0.000000] psci: Using standard PSCI v0.2 function IDs52gitlab # [ 0.000000] psci: Trusted OS migration not required53gitlab # [ 0.000000] psci: SMC Calling Convention v1.154gitlab # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)55gitlab # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u31129656gitlab # [ 0.000000] Detected PIPT I-cache on CPU057gitlab # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)58gitlab # [ 0.000000] CPU features: detected: GICv3 CPU interface59gitlab # [ 0.000000] CPU features: detected: Spectre-v460gitlab # [ 0.000000] CPU features: detected: Spectre-BHB61gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_3862gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_2363gitlab # [ 0.000000] alternatives: applying boot alternatives64gitlab # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/h731ginrbpnw32c9g4bw090lw379md9c-nixos-system-gitlab-test/init regInfo=/nix/store/yy0pb2kka5h98wf4n6x3cg4m38bx873r-closure-info/registration console=ttyAMA0,115200n8 console=tty065gitlab # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/yy0pb2kka5h98wf4n6x3cg4m38bx873r-closure-info/registration", will be passed to user space.66gitlab # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes67gitlab # [ 0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)68gitlab # [ 0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)69gitlab # [ 0.000000] software IO TLB: area num 4.70gitlab # [ 0.000000] software IO TLB: mapped [mem 0x00000000fa000000-0x00000000fe000000] (64MB)71gitlab # [ 0.000000] Fallback order for Node 0: 072gitlab # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 157286473gitlab # [ 0.000000] Policy zone: Normal74gitlab # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off75gitlab # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=176gitlab # [ 0.000000] allocated 12582912 bytes of page_ext77gitlab # [ 0.000000] ftrace: allocating 74867 entries in 294 pages78gitlab # [ 0.000000] ftrace: allocated 294 pages with 4 groups79gitlab # [ 0.000000] rcu: Hierarchical RCU implementation.80gitlab # [ 0.000000] rcu: RCU event tracing is enabled.81gitlab # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.82gitlab # [ 0.000000] Trampoline variant of Tasks RCU enabled.83gitlab # [ 0.000000] Rude variant of Tasks RCU enabled.84gitlab # [ 0.000000] Tracing variant of Tasks RCU enabled.85gitlab # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.86gitlab # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=487gitlab # [ 0.000000] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.88gitlab # [ 0.000000] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.89gitlab # [ 0.000000] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.90gitlab # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 091gitlab # [ 0.000000] GICv3: 256 SPIs implemented92gitlab # [ 0.000000] GICv3: 0 Extended SPIs implemented93gitlab # [ 0.000000] Root IRQ handler: gic_handle_irq94gitlab # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI95gitlab # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=096gitlab # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a000097gitlab # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]98gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @100f10000 (indirect, esz 8, psz 64K, shr 1)99gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @100f20000 (flat, esz 8, psz 64K, shr 1)100gitlab # [ 0.000000] GICv3: using LPI property table @0x0000000100f30000101gitlab # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000100f40000102gitlab # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.103gitlab # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns104gitlab # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).105gitlab # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns106gitlab # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns107gitlab # [ 0.000028] arm-pv: using stolen time PV108gitlab # [ 0.000405] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)109gitlab # [ 0.000613] Console: colour dummy device 80x25110gitlab # [ 0.000620] printk: legacy console [tty0] enabled111gitlab # [ 0.000799] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)112gitlab # [ 0.000806] pid_max: default: 32768 minimum: 301113gitlab # [ 0.000892] LSM: initializing lsm=capability,landlock,yama,bpf,ima114gitlab # [ 0.001028] landlock: Up and running.115gitlab # [ 0.001031] Yama: becoming mindful.116gitlab # [ 0.001450] LSM support for eBPF active117gitlab # [ 0.001673] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)118gitlab # [ 0.001788] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)119gitlab # [ 0.002953] cacheinfo: Unable to detect cache hierarchy for CPU 0120gitlab # [ 0.003641] rcu: Hierarchical SRCU implementation.121gitlab # [ 0.003645] rcu: Max phase no-delay instances is 1000.122gitlab # [ 0.003805] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level123gitlab # [ 0.004901] fsl-mc MSI: its@8080000 domain created124gitlab # [ 0.004993] EFI services will not be available.125gitlab # [ 0.005158] smp: Bringing up secondary CPUs ...126gitlab # [ 0.005650] Detected PIPT I-cache on CPU1127gitlab # [ 0.005711] GICv3: CPU1: found redistributor 1 region 0:0x00000000080c0000128gitlab # [ 0.005783] GICv3: CPU1: using allocated LPI pending table @0x0000000100f50000129gitlab # [ 0.005855] CPU1: Booted secondary processor 0x0000000001 [0xc00fac40]130gitlab # [ 0.006687] Detected PIPT I-cache on CPU2131gitlab # [ 0.006751] GICv3: CPU2: found redistributor 2 region 0:0x00000000080e0000132gitlab # [ 0.006824] GICv3: CPU2: using allocated LPI pending table @0x0000000100f60000133gitlab # [ 0.006896] CPU2: Booted secondary processor 0x0000000002 [0xc00fac40]134gitlab # [ 0.007644] Detected PIPT I-cache on CPU3135gitlab # [ 0.007713] GICv3: CPU3: found redistributor 3 region 0:0x0000000008100000136gitlab # [ 0.007785] GICv3: CPU3: using allocated LPI pending table @0x0000000100f70000137gitlab # [ 0.007859] CPU3: Booted secondary processor 0x0000000003 [0xc00fac40]138gitlab # [ 0.008287] smp: Brought up 1 node, 4 CPUs139gitlab # [ 0.008313] SMP: Total of 4 processors activated.140gitlab # [ 0.008315] CPU: All CPU(s) started at EL1141gitlab # [ 0.008324] CPU features: detected: Branch Target Identification142gitlab # [ 0.008328] CPU features: detected: ARMv8.4 Translation Table Level143gitlab # [ 0.008330] CPU features: detected: Instruction cache invalidation not required for I/D coherence144gitlab # [ 0.008334] CPU features: detected: Data cache clean to the PoU not required for I/D coherence145gitlab # [ 0.008337] CPU features: detected: Common not Private translations146gitlab # [ 0.008340] CPU features: detected: CRC32 instructions147gitlab # [ 0.008342] CPU features: detected: Data cache clean to Point of Deep Persistence148gitlab # [ 0.008345] CPU features: detected: Data cache clean to Point of Persistence149gitlab # [ 0.008348] CPU features: detected: Data independent timing control (DIT)150gitlab # [ 0.008351] CPU features: detected: E0PD151gitlab # [ 0.008353] CPU features: detected: Enhanced Counter Virtualization152gitlab # [ 0.008355] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)153gitlab # [ 0.008358] CPU features: detected: Enhanced Virtualization Traps154gitlab # [ 0.008361] CPU features: detected: Fine Grained Traps155gitlab # [ 0.008364] CPU features: detected: Generic authentication (architected QARMA5 algorithm)156gitlab # [ 0.008368] CPU features: detected: RCpc load-acquire (LDAPR)157gitlab # [ 0.008370] CPU features: detected: LSE atomic instructions158gitlab # [ 0.008373] CPU features: detected: Privileged Access Never159gitlab # [ 0.008375] CPU features: detected: PMUv3160gitlab # [ 0.008377] CPU features: detected: RAS Extension Support161gitlab # [ 0.008380] CPU features: detected: RASv1p1 Extension Support162gitlab # [ 0.008382] CPU features: detected: Random Number Generator163gitlab # [ 0.008384] CPU features: detected: Speculation barrier (SB)164gitlab # [ 0.008387] CPU features: detected: Stage-2 Force Write-Back165gitlab # [ 0.008389] CPU features: detected: TLB range maintenance instructions166gitlab # [ 0.008393] CPU features: detected: Speculative Store Bypassing Safe (SSBS)167gitlab # [ 0.008480] alternatives: applying system-wide alternatives168gitlab # [ 0.011309] CPU features: detected: BBM Level 2 without TLB conflict abort169gitlab # [ 0.011547] Memory: 5963732K/6291456K available (24448K kernel code, 7090K rwdata, 26340K rodata, 4736K init, 1106K bss, 272908K reserved, 32768K cma-reserved)170gitlab # [ 0.012632] devtmpfs: initialized171gitlab # [ 0.015987] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)172gitlab # [ 0.016053] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).173gitlab # [ 0.016287] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL174gitlab # [ 0.016292] 0 pages in range for non-PLT usage175gitlab # [ 0.016293] 508288 pages in range for PLT usage176gitlab # [ 0.016413] pinctrl core: initialized pinctrl subsystem177gitlab # [ 0.017188] DMI not present or invalid.178gitlab # [ 0.020439] NET: Registered PF_NETLINK/PF_ROUTE protocol family179gitlab # [ 0.022865] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations180gitlab # [ 0.023107] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations181gitlab # [ 0.025354] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations182gitlab # [ 0.025376] audit: initializing netlink subsys (disabled)183gitlab # [ 0.025555] audit: type=2000 audit(0.024:1): state=initialized audit_enabled=0 res=1184gitlab # [ 0.026032] thermal_sys: Registered thermal governor 'fair_share'185gitlab # [ 0.026034] thermal_sys: Registered thermal governor 'bang_bang'186gitlab # [ 0.026038] thermal_sys: Registered thermal governor 'step_wise'187gitlab # [ 0.026041] thermal_sys: Registered thermal governor 'user_space'188gitlab # [ 0.026043] thermal_sys: Registered thermal governor 'power_allocator'189gitlab # [ 0.026097] cpuidle: using governor ladder190gitlab # [ 0.026115] cpuidle: using governor menu191gitlab # [ 0.026299] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.192gitlab # [ 0.026426] ASID allocator initialised with 65536 entries193gitlab # [ 0.027488] Serial: AMBA PL011 UART driver194gitlab # [ 0.032543] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1195gitlab # [ 0.032757] printk: console [ttyAMA0] enabled196gitlab # [ 0.038584] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages197gitlab # [ 0.038596] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page198gitlab # [ 0.038601] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages199gitlab # [ 0.038606] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page200gitlab # [ 0.038610] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages201gitlab # [ 0.038614] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page202gitlab # [ 0.038618] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages203gitlab # [ 0.038622] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page204gitlab # [ 0.044109] fbcon: Taking over console205gitlab # [ 0.044127] ACPI: Interpreter disabled.206gitlab # [ 0.045501] iommu: Default domain type: Translated207gitlab # [ 0.045508] iommu: DMA domain TLB invalidation policy: strict mode208gitlab # [ 0.061840] SCSI subsystem initialized209gitlab # [ 0.062540] usbcore: registered new interface driver usbfs210gitlab # [ 0.062589] usbcore: registered new interface driver hub211gitlab # [ 0.062611] usbcore: registered new device driver usb212gitlab # [ 0.063048] pps_core: LinuxPPS API ver. 1 registered213gitlab # [ 0.063054] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>214gitlab # [ 0.063065] PTP clock support registered215gitlab # [ 0.063120] EDAC MC: Ver: 3.0.0216gitlab # [ 0.063435] scmi_core: SCMI protocol bus registered217gitlab # [ 0.064044] FPGA manager framework218gitlab # [ 0.065305] vgaarb: loaded219gitlab # [ 0.067236] clocksource: Switched to clocksource arch_sys_counter220gitlab # [ 0.069702] VFS: Disk quotas dquot_6.6.0221gitlab # [ 0.069737] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)222gitlab # [ 0.070240] netfs: FS-Cache loaded223gitlab # [ 0.070401] pnp: PnP ACPI: disabled224gitlab # [ 0.075524] NET: Registered PF_INET protocol family225gitlab # [ 0.076996] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)226gitlab # [ 0.118933] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)227gitlab # [ 0.118998] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)228gitlab # [ 0.119054] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)229gitlab # [ 0.119465] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)230gitlab # [ 0.120236] TCP: Hash tables configured (established 65536 bind 65536)231gitlab # [ 0.120348] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)232gitlab # [ 0.120422] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)233gitlab # [ 0.120567] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)234gitlab # [ 0.120769] NET: Registered PF_UNIX/PF_LOCAL protocol family235gitlab # [ 0.120795] NET: Registered PF_XDP protocol family236gitlab # [ 0.120813] PCI: CLS 0 bytes, default 64237gitlab # [ 0.120987] Trying to unpack rootfs image as initramfs...238gitlab # [ 0.131714] kvm [1]: HYP mode not available239gitlab # [ 0.195974] Initialise system trusted keyrings240gitlab # [ 0.196245] workingset: timestamp_bits=42 max_order=21 bucket_order=0241gitlab # [ 0.196871] squashfs: version 4.0 (2009/01/31) Phillip Lougher242gitlab # [ 0.196934] 9p: Installing v9fs 9p2000 file system support243gitlab # [ 0.211694] Key type asymmetric registered244gitlab # [ 0.211700] Asymmetric key parser 'x509' registered245gitlab # [ 0.211761] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)246gitlab # [ 0.211910] io scheduler mq-deadline registered247gitlab # [ 0.211916] io scheduler kyber registered248gitlab # [ 0.220516] pl061_gpio 9030000.pl061: PL061 GPIO chip registered249gitlab # [ 0.223809] ledtrig-cpu: registered to indicate activity on CPUs250gitlab # [ 0.224464] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:251gitlab # [ 0.224485] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000252gitlab # [ 0.224497] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000253gitlab # [ 0.224505] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000254gitlab # [ 0.224531] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits255gitlab # [ 0.224558] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]256gitlab # [ 0.224659] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00257gitlab # [ 0.224669] pci_bus 0000:00: root bus resource [bus 00-ff]258gitlab # [ 0.224674] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]259gitlab # [ 0.224678] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]260gitlab # [ 0.224682] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]261gitlab # [ 0.224777] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint262gitlab # [ 0.225471] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint263gitlab # [ 0.225781] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]264gitlab # [ 0.225798] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]265gitlab # [ 0.225838] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]266gitlab # [ 0.225853] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]267gitlab # [ 0.226548] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint268gitlab # [ 0.226946] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]269gitlab # [ 0.226983] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]270gitlab # [ 0.227042] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]271gitlab # [ 0.228050] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint272gitlab # [ 0.228365] pci 0000:00:03.0: BAR 0 [io 0x0000-0x001f]273gitlab # [ 0.228383] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]274gitlab # [ 0.228432] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]275gitlab # [ 0.229311] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint276gitlab # [ 0.229630] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]277gitlab # [ 0.229649] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]278gitlab # [ 0.229696] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]279gitlab # [ 0.230553] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint280gitlab # [ 0.230851] pci 0000:00:05.0: BAR 0 [io 0x0000-0x007f]281gitlab # [ 0.230868] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]282gitlab # [ 0.230915] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]283gitlab # [ 0.231789] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint284gitlab # [ 0.232158] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]285gitlab # [ 0.232174] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]286gitlab # [ 0.232220] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]287gitlab # [ 0.233086] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint288gitlab # [ 0.233450] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]289gitlab # [ 0.233473] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]290gitlab # [ 0.233517] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]291gitlab # [ 0.233540] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]292gitlab # [ 0.234360] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint293gitlab # [ 0.234650] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]294gitlab # [ 0.234690] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]295gitlab # [ 0.235593] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint296gitlab # [ 0.235994] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]297gitlab # [ 0.236031] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]298gitlab # [ 0.236733] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint299gitlab # [ 0.237150] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]300gitlab # [ 0.237515] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint301gitlab # [ 0.237957] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]302gitlab # [ 0.237975] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]303gitlab # [ 0.238010] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]304gitlab # [ 0.238769] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint305gitlab # [ 0.239053] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]306gitlab # [ 0.239069] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]307gitlab # [ 0.239105] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]308gitlab # [ 0.239970] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned309gitlab # [ 0.239979] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned310gitlab # [ 0.239984] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned311gitlab # [ 0.240053] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned312gitlab # [ 0.240118] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned313gitlab # [ 0.240189] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned314gitlab # [ 0.240256] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned315gitlab # [ 0.240328] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned316gitlab # [ 0.240388] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned317gitlab # [ 0.240457] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned318gitlab # [ 0.240516] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned319gitlab # [ 0.240592] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned320gitlab # [ 0.240685] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned321gitlab # [ 0.240753] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned322gitlab # [ 0.240781] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned323gitlab # [ 0.240804] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned324gitlab # [ 0.240838] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned325gitlab # [ 0.240861] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned326gitlab # [ 0.240884] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned327gitlab # [ 0.240909] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned328gitlab # [ 0.240932] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned329gitlab # [ 0.240960] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned330gitlab # [ 0.240984] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned331gitlab # [ 0.241018] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned332gitlab # [ 0.241043] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned333gitlab # [ 0.241066] pci 0000:00:05.0: BAR 0 [io 0x1000-0x107f]: assigned334gitlab # [ 0.241089] pci 0000:00:06.0: BAR 0 [io 0x1080-0x10ff]: assigned335gitlab # [ 0.241111] pci 0000:00:0b.0: BAR 0 [io 0x1100-0x113f]: assigned336gitlab # [ 0.241139] pci 0000:00:01.0: BAR 0 [io 0x1140-0x115f]: assigned337gitlab # [ 0.241162] pci 0000:00:02.0: BAR 0 [io 0x1160-0x117f]: assigned338gitlab # [ 0.241184] pci 0000:00:03.0: BAR 0 [io 0x1180-0x119f]: assigned339gitlab # [ 0.241208] pci 0000:00:04.0: BAR 0 [io 0x11a0-0x11bf]: assigned340gitlab # [ 0.241235] pci 0000:00:07.0: BAR 0 [io 0x11c0-0x11df]: assigned341gitlab # [ 0.241263] pci 0000:00:0c.0: BAR 0 [io 0x11e0-0x11ff]: assigned342gitlab # [ 0.241291] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]343gitlab # [ 0.241297] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]344gitlab # [ 0.241301] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]345gitlab # [ 0.243212] pci 0000:00:0a.0: enabling device (0000 -> 0002)346gitlab # [ 0.258204] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)347gitlab # [ 0.260829] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)348gitlab # [ 0.263460] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)349gitlab # [ 0.266250] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)350gitlab # [ 0.269282] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)351gitlab # [ 0.272671] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)352gitlab # [ 0.275436] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)353gitlab # [ 0.277896] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)354gitlab # [ 0.280364] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)355gitlab # [ 0.284945] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)356gitlab # [ 0.287623] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)357gitlab # [ 0.293719] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled358gitlab # [ 0.295664] msm_serial: driver initialized359gitlab # [ 0.295838] SuperH (H)SCI(F) driver initialized360gitlab # [ 0.295896] STM32 USART driver initialized361gitlab # [ 0.316166] loop: module loaded362gitlab # [ 0.316381] virtio_blk virtio4: 4/0/0 default/read/poll queues363gitlab # [ 0.318351] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)364gitlab # [ 0.320534] virtio_blk virtio5: 4/0/0 default/read/poll queues365gitlab # [ 0.322295] virtio_blk virtio5: [vdb] 12395768 512-byte logical blocks (6.35 GB/5.91 GiB)366gitlab # [ 0.324427] megasas: 07.734.00.00-rc1367gitlab # [ 0.325084] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]368gitlab # [ 0.326916] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000369gitlab # [ 0.326979] Intel/Sharp Extended Query Table at 0x0031370gitlab # [ 0.328768] Using buffer write method371gitlab # [ 0.328797] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]372gitlab # [ 0.330504] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000373gitlab # [ 0.330517] Intel/Sharp Extended Query Table at 0x0031374gitlab # [ 0.332358] Using buffer write method375gitlab # [ 0.332379] Concatenating MTD devices:376gitlab # [ 0.332383] (0): "0.flash"377gitlab # [ 0.332386] (1): "0.flash"378gitlab # [ 0.332389] into device "0.flash"379gitlab # [ 0.532881] Freeing initrd memory: 27184K380gitlab # [ 0.542328] tun: Universal TUN/TAP device driver, 1.6381gitlab # [ 0.547352] thunder_xcv, ver 1.0382gitlab # [ 0.547408] thunder_bgx, ver 1.0383gitlab # [ 0.547437] nicpf, ver 1.0384gitlab # [ 0.548328] e1000: Intel(R) PRO/1000 Network Driver385gitlab # [ 0.548335] e1000: Copyright (c) 1999-2006 Intel Corporation.386gitlab # [ 0.548379] e1000e: Intel(R) PRO/1000 Network Driver387gitlab # [ 0.548386] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.388gitlab # [ 0.548420] igb: Intel(R) Gigabit Ethernet Network Driver389gitlab # [ 0.548425] igb: Copyright (c) 2007-2014 Intel Corporation.390gitlab # [ 0.548454] igbvf: Intel(R) Gigabit Virtual Function Network Driver391gitlab # [ 0.548459] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.392gitlab # [ 0.548677] sky2: driver version 1.30393gitlab # [ 0.551161] usbcore: registered new interface driver usb-storage394gitlab # [ 0.551330] usbcore: registered new interface driver usbserial_generic395gitlab # [ 0.551348] usbserial: USB Serial support registered for generic396gitlab # [ 0.551594] ehci-pci 0000:00:0a.0: EHCI Host Controller397gitlab # [ 0.551630] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 1398gitlab # [ 0.552041] ehci-pci 0000:00:0a.0: irq 15, io mem 0x10089000399gitlab # [ 0.552316] hv_vmbus: registering driver hyperv_keyboard400gitlab # [ 0.553812] rtc-pl031 9010000.pl031: registered as rtc0401gitlab # [ 0.553864] rtc-pl031 9010000.pl031: setting system clock to 2026-11-13T10:33:29 UTC (1794566009)402gitlab # [ 0.554426] i2c_dev: i2c /dev entries driver403gitlab # [ 0.558701] sdhci: Secure Digital Host Controller Interface driver404gitlab # [ 0.558708] sdhci: Copyright(c) Pierre Ossman405gitlab # [ 0.559111] Synopsys Designware Multimedia Card Interface Driver406gitlab # [ 0.559722] sdhci-pltfm: SDHCI platform and OF driver helper407gitlab # [ 0.562229] hid: raw HID events driver (C) Jiri Kosina408gitlab # [ 0.562642] usbcore: registered new interface driver usbhid409gitlab # [ 0.562648] usbhid: USB HID core driver410gitlab # [ 0.563330] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.00411gitlab # [ 0.563813] hub 1-0:1.0: USB hub found412gitlab # [ 0.563830] hub 1-0:1.0: 6 ports detected413gitlab # [ 0.596615] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available414gitlab # [ 0.599015] drop_monitor: Initializing network drop monitor service415gitlab # [ 0.599262] NET: Registered PF_INET6 protocol family416gitlab # [ 0.600323] Segment Routing with IPv6417gitlab # [ 0.600341] In-situ OAM (IOAM) with IPv6418gitlab # [ 0.600396] NET: Registered PF_PACKET protocol family419gitlab # [ 0.600564] 9pnet: Installing 9P2000 support420gitlab # [ 0.602854] Key type dns_resolver registered421gitlab # [ 0.612222] registered taskstats version 1422gitlab # [ 0.612533] Loading compiled-in X.509 certificates423gitlab # [ 0.627348] Demotion targets for Node 0: null424gitlab # [ 0.627669] Key type .fscrypt registered425gitlab # [ 0.627675] Key type fscrypt-provisioning registered426gitlab # [ 0.627875] ima: No TPM chip found, activating TPM-bypass!427gitlab # [ 0.627891] ima: Allocated hash algorithm: sha1428gitlab # [ 0.628087] ima: No architecture policies found429gitlab # [ 0.629167] input: gpio-keys as /devices/platform/gpio-keys/input/input0430gitlab # [ 0.653198] clk: Disabling unused clocks431gitlab # [ 0.653232] PM: genpd: Disabling unused power domains432gitlab # [ 0.657864] Freeing unused kernel memory: 4736K433gitlab # [ 0.658089] Run /init as init process434gitlab # [ 0.682769] systemd[1]: Successfully made /usr/ read-only.435gitlab # [ 0.819279] usb 1-1: new high-speed USB device number 2 using ehci-pci436gitlab # [ 0.969129] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1437gitlab # [ 1.016943] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)438gitlab # [ 1.016983] systemd[1]: Detected virtualization qemu.439gitlab # [ 1.017025] systemd[1]: Detected architecture arm64.440gitlab # [ 1.017040] systemd[1]: Running in initrd.441gitlab # [ 1.017494] systemd[1]: Initializing machine ID from random generator.442gitlab # [ 1.017715] systemd[1]: Hostname set to <gitlab>.443gitlab # [ 1.083580] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input0444gitlab # [ 1.203308] usb 1-2: new high-speed USB device number 3 using ehci-pci445gitlab # [ 1.211211] systemd[1]: bpf-restrict-fs: LSM BPF program attached446gitlab # [ 1.273972] systemd[1]: Queued start job for default target Initrd Default Target.447gitlab # [ 1.298210] systemd[1]: Created slice Slice /system/modprobe.448gitlab # [ 1.298461] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.449gitlab # [ 1.298497] systemd[1]: Expecting device /dev/disk/by-label/nix-store...450gitlab # [ 1.298511] systemd[1]: Expecting device /dev/disk/by-label/nixos...451gitlab # [ 1.298534] systemd[1]: Reached target Path Units.452gitlab # [ 1.298551] systemd[1]: Reached target Slice Units.453gitlab # [ 1.298570] systemd[1]: Reached target Swaps.454gitlab # [ 1.298588] systemd[1]: Reached target Timer Units.455gitlab # [ 1.298754] systemd[1]: Listening on D-Bus System Message Bus Socket.456gitlab # [ 1.298926] systemd[1]: Listening on Journal Socket (/dev/log).457gitlab # [ 1.299075] systemd[1]: Listening on Journal Sockets.458gitlab # [ 1.299218] systemd[1]: Listening on udev Control Socket.459gitlab # [ 1.299460] systemd[1]: Listening on udev Kernel Socket.460gitlab # [ 1.299482] systemd[1]: Reached target Socket Units.461gitlab # [ 1.301137] systemd[1]: Starting Create List of Static Device Nodes...462gitlab # [ 1.302825] systemd[1]: Starting Load Kernel Module 9pnet_virtio...463gitlab # [ 1.302895] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs464gitlab # [ 1.304638] systemd[1]: Mounting Kernel Configuration File System...465gitlab # [ 1.321079] systemd[1]: Starting Journal Service...466gitlab # [ 1.323228] systemd[1]: Starting Load Kernel Modules...467gitlab # [ 1.323395] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os468gitlab # [ 1.327509] systemd[1]: Starting Coldplug All udev Devices...469gitlab # [ 1.332563] systemd[1]: Finished Create List of Static Device Nodes.470gitlab # [ 1.333723] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.471gitlab # [ 1.334183] systemd[1]: Finished Load Kernel Module 9pnet_virtio.472gitlab # [ 1.334560] systemd[1]: Mounted Kernel Configuration File System.473gitlab # [ 1.336749] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...474gitlab # [ 1.355948] systemd-journald[96]: Collecting audit messages is disabled.475gitlab # [ 1.356021] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.476gitlab # [ 1.356151] [drm] pci: virtio-gpu-pci detected at 0000:00:08.0477gitlab # [ 1.356296] [drm] features: -virgl +edid -resource_blob -host_visible478gitlab # [ 1.356299] [drm] features: -context_init479gitlab # [ 1.357794] [drm] number of scanouts: 1480gitlab # [ 1.357819] [drm] number of cap sets: 0481gitlab # [ 1.360655] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev482gitlab # [ 1.360828] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.483gitlab # [ 1.363489] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic484gitlab # [ 1.363500] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 0485gitlab # [ 1.367475] systemd[1]: Starting Create Static Device Nodes in /dev...486gitlab # [ 1.373538] Console: switching to colour frame buffer device 160x50487gitlab # [ 1.379646] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device488gitlab # [ 1.388848] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2489gitlab # [ 1.392330] systemd[1]: Finished Create Static Device Nodes in /dev.490gitlab # [ 1.392352] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input0491gitlab # [ 1.393155] systemd[1]: Reached target Preparation for Local File Systems.492gitlab # [ 1.393202] systemd[1]: Reached target Local File Systems.493gitlab # [ 1.407375] systemd[1]: Starting Rule-based Manager for Device Events and Files...494gitlab # [ 1.408047] systemd[1]: Finished Load Kernel Modules.495gitlab # [ 1.409397] systemd[1]: Starting Apply Kernel Variables...496gitlab # [ 1.429777] systemd-modules-load[97]: Using 4 probe threads497gitlab # [ 1.431162] systemd-modules-load[97]: Module 'virtio_balloon' is built in498gitlab # [ 1.435660] systemd[1]: Started Journal Service.499gitlab # [ 1.435620] systemd-modules-load[97]: Module 'virtio_console' is built in500gitlab # [ 1.437138] systemd-modules-load[97]: Module 'virtio_rng' is built in501gitlab # [ 1.438411] systemd-modules-load[97]: Inserted module 'dm_mod'502gitlab # [ 1.439663] systemd-modules-load[97]: Inserted module 'virtio_gpu'503gitlab # [ 1.441047] systemd[1]: Starting Create System Files and Directories...504gitlab # [ 1.442764] systemd[1]: Finished Apply Kernel Variables.505gitlab # [ 1.455578] systemd[1]: Finished Create System Files and Directories.506gitlab # [ 1.458773] systemd-udevd[106]: Using default interface naming scheme 'v261'.507gitlab # [ 1.472088] systemd[1]: Started Rule-based Manager for Device Events and Files.508gitlab # [ 1.493016] systemd[1]: Starting Virtual Console Setup...509gitlab # [ 1.533755] systemd-vconsole-setup[130]: Configuration of first virtual console was skipped, ignoring remaining ones.510gitlab # [ 1.536441] systemd[1]: Finished Virtual Console Setup.511gitlab # [ 1.948249] systemd[1]: Finished Coldplug All udev Devices.512gitlab # [ 1.949705] systemd[1]: Reached target System Initialization.513gitlab # [ 1.951015] systemd[1]: Reached target Basic System.514gitlab # [ 2.068823] (udev-worker)[120]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.515gitlab # [ 2.072602] (udev-worker)[120]: Network interface NamePolicy= disabled on kernel command line.516gitlab # [ 2.074926] (udev-worker)[119]: Network interface NamePolicy= disabled on kernel command line.517gitlab # [ 2.085672] systemd[1]: Found device /dev/disk/by-label/nixos.518gitlab # [ 2.088193] systemd[1]: Reached target Initrd Root Device.519gitlab # [ 2.089715] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...520gitlab # [ 2.107377] systemd[1]: Found device /dev/disk/by-label/nix-store.521gitlab # [ 2.132815] systemd-fsck[152]: nixos: clean, 12/65536 files, 13019/262144 blocks522gitlab # [ 2.135224] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.523gitlab # [ 2.327434] systemd[1]: Mounting /sysroot...524gitlab # [ 2.365480] EXT4-fs (vda): mounted filesystem f0fd218e-56af-473f-84ce-fdcfeaf8fa93 r/w with ordered data mode. Quota mode: none.525gitlab # [ 2.365435] systemd[1]: Mounted /sysroot.526gitlab # [ 2.366448] systemd[1]: Reached target Initrd Root File System.527gitlab # [ 2.368781] systemd[1]: Mounting /sysroot/nix/.ro-store...528gitlab # [ 2.371761] systemd[1]: Mounting /sysroot/nix/.rw-store...529gitlab # [ 2.373310] systemd[1]: Mounting /sysroot/run...530gitlab # [ 2.388258] systemd[1]: Mounting /sysroot/tmp/shared...531gitlab # [ 2.392250] systemd[1]: Mounting /sysroot/tmp/xchg...532gitlab # [ 2.400509] systemd[1]: Starting Mountpoints Configured in the Real Root...533gitlab # [ 2.419618] systemd-sysroot-fstab-check[166]: /sysroot should be mounted in the initrd, will request daemon-reload.534gitlab # [ 2.435023] systemd[1]: Mounted /sysroot/nix/.rw-store.[ 2.438684] erofs (device vdb): mounted with root inode @ nid 128.535gitlab # 536gitlab # [ 2.437964] systemd[1]: Mounted /sysroot/run.537gitlab # [ 2.444186] systemd[1]: Mounted /sysroot/nix/.ro-store.538gitlab # [ 2.445133] systemd[1]: Mounted /sysroot/tmp/shared.539gitlab # [ 2.445864] systemd[1]: Mounted /sysroot/tmp/xchg.540gitlab # [ 2.448490] systemd[1]: Starting rw-sysroot-nix-store.service...541gitlab # [ 2.451926] systemd[1]: Reload requested from client PID 166 ('systemd-sysroot') (unit initrd-parse-etc.service)...542gitlab # [ 2.453796] systemd[1]: Reloading...543gitlab # [ 2.549413] systemd[1]: Reloading finished in 98 ms.544gitlab # [ 2.591124] (udev-worker)[138]: mtd0ro: Failed to find and pin callout binary "/nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/mtd_probe": No such file or directory545gitlab # [ 2.595100] (udev-worker)[138]: mtd0ro: /nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory546gitlab # [ 2.610835] systemd-sysroot-fstab-check[166]: Requesting initrd-fs.target/start/replace...547gitlab # [ 2.613323] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.548gitlab # [ 2.614400] systemd[1]: Finished rw-sysroot-nix-store.service.549gitlab # [ 2.617969] systemd-sysroot-fstab-check[166]: Requesting swap.target/start/replace...550gitlab # [ 2.624296] systemd[1]: initrd-parse-etc.service: Deactivated successfully.551gitlab # [ 2.625314] systemd[1]: Finished Mountpoints Configured in the Real Root.552gitlab # [ 2.626262] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.553gitlab # [ 2.631914] systemd[1]: Starting rw-sysroot-nix-store.service...554gitlab # [ 2.633073] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.555gitlab # [ 2.634175] systemd[1]: Stopped Virtual Console Setup.556gitlab # [ 2.634980] systemd[1]: Stopping Virtual Console Setup...557gitlab # [ 2.636502] systemd[1]: Starting Virtual Console Setup...558gitlab # [ 2.647512] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.559gitlab # [ 2.648760] systemd[1]: Finished rw-sysroot-nix-store.service.560gitlab # [ 2.659225] systemd-vconsole-setup[194]: Configuration of first virtual console was skipped, ignoring remaining ones.561gitlab # [ 2.661541] systemd[1]: Finished Virtual Console Setup.562gitlab # [ 3.329861] systemd[1]: Mounting /sysroot/nix/store...563gitlab # [ 3.369414] systemd[1]: Mounted /sysroot/nix/store.564gitlab # [ 3.370667] systemd[1]: Reached target Initrd File Systems.565gitlab # [ 3.371920] systemd[1]: Starting Find NixOS closure...566gitlab # [ 3.373493] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...567gitlab # [ 3.410482] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.568gitlab # [ 3.412404] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.569gitlab # [ 3.419931] systemd[1]: Finished Find NixOS closure.570gitlab # [ 3.421217] systemd[1]: Reached target Initrd Default Target.571gitlab # [ 3.422509] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...572gitlab # [ 3.443044] systemd[1]: Stopped target Initrd Default Target.573gitlab # [ 3.444393] systemd[1]: Stopped target Basic System.574gitlab # [ 3.445526] systemd[1]: Stopped target Initrd Root Device.575gitlab # [ 3.446745] systemd[1]: Stopped target Path Units.576gitlab # [ 3.447834] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.577gitlab # [ 3.449623] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.578gitlab # [ 3.451308] systemd[1]: Stopped target Slice Units.579gitlab # [ 3.452510] systemd[1]: Stopped target Socket Units.580gitlab # [ 3.455506] systemd[1]: Stopped target System Initialization.581gitlab # [ 3.456924] systemd[1]: Stopped target Swaps.582gitlab # [ 3.458117] systemd[1]: Stopped target Timer Units.583gitlab # [ 3.459371] systemd[1]: dbus.socket: Deactivated successfully.584gitlab # [ 3.463908] systemd[1]: Closed D-Bus System Message Bus Socket.585gitlab # [ 3.465447] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.586gitlab # [ 3.467083] systemd[1]: Stopped Find NixOS closure.587gitlab # [ 3.468271] systemd[1]: Starting Load Kernel Module 9pnet_virtio...588gitlab # [ 3.469708] systemd[1]: Starting rw-sysroot-nix-store.service...589gitlab # [ 3.471041] systemd[1]: systemd-sysctl.service: Deactivated successfully.590gitlab # [ 3.472606] systemd[1]: Stopped Apply Kernel Variables.591gitlab # [ 3.473782] systemd[1]: systemd-modules-load.service: Deactivated successfully.592gitlab # [ 3.475333] systemd[1]: Stopped Load Kernel Modules.593gitlab # [ 3.476486] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.594gitlab # [ 3.478252] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.595gitlab # [ 3.479930] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.596gitlab # [ 3.481966] systemd[1]: Stopped Create System Files and Directories.597gitlab # [ 3.483339] systemd[1]: Stopped target Local File Systems.598gitlab # [ 3.484670] systemd[1]: Stopped target Preparation for Local File Systems.599gitlab # [ 3.486209] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.600gitlab # [ 3.487726] systemd[1]: Stopped Coldplug All udev Devices.601gitlab # [ 3.489009] systemd[1]: Stopping Rule-based Manager for Device Events and Files...602gitlab # [ 3.490530] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.603gitlab # [ 3.492047] systemd[1]: Stopped Virtual Console Setup.604gitlab # [ 3.493187] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.605gitlab # [ 3.494594] systemd[1]: Finished Load Kernel Module 9pnet_virtio.606gitlab # [ 3.495841] systemd[1]: initrd-cleanup.service: Deactivated successfully.607gitlab # [ 3.497243] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.608gitlab # [ 3.498492] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.609gitlab # [ 3.499851] systemd[1]: Finished rw-sysroot-nix-store.service.610gitlab # [ 3.501083] systemd[1]: systemd-udevd.service: Deactivated successfully.611gitlab # [ 3.502331] systemd[1]: Stopped Rule-based Manager for Device Events and Files.612gitlab # [ 3.503682] systemd[1]: systemd-udevd.service: Consumed 1.909s CPU time over 2.081s wall clock time, 31.6M memory peak.613gitlab # [ 3.505772] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.614gitlab # [ 3.507170] systemd[1]: Closed udev Control Socket.615gitlab # [ 3.508330] systemd[1]: Starting Cleanup udev Database...616gitlab # [ 3.509453] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.617gitlab # [ 3.510906] systemd[1]: Stopped Create Static Device Nodes in /dev.618gitlab # [ 3.512126] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.619gitlab # [ 3.513674] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.620gitlab # [ 3.515027] systemd[1]: kmod-static-nodes.service: Deactivated successfully.621gitlab # [ 3.516401] systemd[1]: Stopped Create List of Static Device Nodes.622gitlab # [ 3.557403] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.623gitlab # [ 3.559046] systemd[1]: Finished Cleanup udev Database.624gitlab # [ 3.560325] systemd[1]: Reached target Switch Root.625gitlab # [ 3.561430] systemd[1]: Starting NixOS Activation...626gitlab # [ 3.684260] initrd-nixos-activation-start[221]: booting system configuration /nix/store/h731ginrbpnw32c9g4bw090lw379md9c-nixos-system-gitlab-test627gitlab # [ 3.717890] initrd-nixos-activation-start[221]: running activation script...628gitlab # [ 3.932104] initrd-nixos-activation-start[245]: setting up /etc...629gitlab # [ 4.034573] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.630gitlab # [ 4.036422] systemd[1]: Finished NixOS Activation.631gitlab # [ 4.037632] systemd[1]: Starting Switch Root...632gitlab # [ 4.066615] systemd[1]: Switching root.633gitlab # [ 4.209638] systemd-journald[96]: Received SIGTERM from PID 1 (systemd).634gitlab # [ 4.766979] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)635gitlab # [ 4.767061] systemd[1]: Detected virtualization qemu.636gitlab # [ 4.767118] systemd[1]: Detected architecture arm64.637gitlab # [ 4.767224] systemd[1]: Detected first boot.638gitlab # [ 4.768001] systemd[1]: Initializing machine ID from random generator.639gitlab # [ 4.977311] systemd[1]: bpf-restrict-fs: LSM BPF program attached640gitlab # [ 5.055716] systemd[1]: Applying preset policy.641gitlab # [ 5.218393] systemd[1]: Populated /etc with preset unit settings.642gitlab # [ 5.361043] systemd[1]: initrd-switch-root.service: Deactivated successfully.643gitlab # [ 5.361294] systemd[1]: Stopped initrd-switch-root.service.644gitlab # [ 5.363036] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.645gitlab # [ 5.366276] systemd[1]: Created slice Slice /system/getty.646gitlab # [ 5.368370] systemd[1]: Created slice GitLab DevOps Platform Slice.647gitlab # [ 5.370467] systemd[1]: Created slice User and Session Slice.648gitlab # [ 5.371036] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.649gitlab # [ 5.372318] systemd[1]: Started Forward Password Requests to Wall Directory Watch.650gitlab # [ 5.372897] systemd[1]: Expecting device /dev/hvc0...651gitlab # [ 5.373295] systemd[1]: Expecting device /dev/ttyAMA0...652gitlab # [ 5.373688] systemd[1]: Reached target Local Encrypted Volumes.653gitlab # [ 5.374085] systemd[1]: Stopped target initrd-fs.target.654gitlab # [ 5.374477] systemd[1]: Stopped target initrd-root-fs.target.655gitlab # [ 5.374856] systemd[1]: Stopped target initrd-switch-root.target.656gitlab # [ 5.375389] systemd[1]: Reached target Virtual Machines and Containers.657gitlab # [ 5.376022] systemd[1]: Reached target Path Units.658gitlab # [ 5.376647] systemd[1]: Reached target Remote File Systems.659gitlab # [ 5.377241] systemd[1]: Reached target Slice Units.660gitlab # [ 5.377843] systemd[1]: Reached target Swaps.661gitlab # [ 5.379984] systemd[1]: Listening on Query the User Interactively for a Password.662gitlab # [ 5.382240] systemd[1]: Listening on Process Core Dump Socket.663gitlab # [ 5.383904] systemd[1]: Listening on Credential Encryption/Decryption.664gitlab # [ 5.385820] systemd[1]: Listening on Factory Reset Management.665gitlab # [ 5.386557] systemd[1]: Listening on Hostname Service Socket.666gitlab # [ 5.391291] systemd[1]: Starting Journal Log Access Socket...667gitlab # [ 5.392499] systemd[1]: Listening on Journal Audit Socket.668gitlab # [ 5.393959] systemd[1]: Listening on Console Output Muting Service Socket.669gitlab # [ 5.394412] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.670gitlab # [ 5.394706] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os671gitlab # [ 5.394943] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki672gitlab # [ 5.396977] systemd[1]: Listening on Disk Repartitioning Service Socket.673gitlab # [ 5.397481] systemd[1]: Listening on udev Control Socket.674gitlab # [ 5.397992] systemd[1]: Listening on udev Varlink Socket.675gitlab # [ 5.402391] systemd[1]: Mounting Huge Pages File System...676gitlab # [ 5.405928] systemd[1]: Mounting POSIX Message Queue File System...677gitlab # [ 5.411546] systemd[1]: Mounting Kernel Debug File System...678gitlab # [ 5.416924] systemd[1]: Mounting Kernel Trace File System...679gitlab # [ 5.423519] systemd[1]: Starting Create List of Static Device Nodes...680gitlab # [ 5.428969] systemd[1]: Starting Load Kernel Module 9pnet_virtio...681gitlab # [ 5.430998] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs682gitlab # [ 5.443750] systemd[1]: Mounting Kernel Configuration File System...683gitlab # [ 5.444105] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm684gitlab # [ 5.444347] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore685gitlab # [ 5.450559] systemd[1]: Starting Load Kernel Module fuse...686gitlab # [ 5.451059] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67687gitlab # [ 5.457233] systemd[1]: Starting Journal Service...688gitlab # [ 5.459633] systemd[1]: Starting Load Kernel Modules...689gitlab # [ 5.465694] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...690gitlab # [ 5.471475] systemd[1]: Starting Remount Root and Kernel File Systems...691gitlab # [ 5.472217] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os692gitlab # [ 5.476790] fuse: init (API version 7.45)693gitlab # [ 5.477360] systemd[1]: Starting Coldplug All udev Devices...694gitlab # [ 5.482510] systemd[1]: Listening on Journal Log Access Socket.695gitlab # [ 5.483668] systemd[1]: Mounted Huge Pages File System.696gitlab # [ 5.487662] systemd[1]: Mounted POSIX Message Queue File System.697gitlab # [ 5.488429] systemd[1]: Mounted Kernel Debug File System.698gitlab # [ 5.491756] systemd[1]: Mounted Kernel Trace File System.699gitlab # [ 5.492879] systemd[1]: Finished Create List of Static Device Nodes.700gitlab # [ 5.494066] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.701gitlab # [ 5.494887] systemd[1]: Finished Load Kernel Module 9pnet_virtio.702gitlab # [ 5.495629] systemd[1]: Mounted Kernel Configuration File System.703gitlab # [ 5.496452] systemd[1]: modprobe@fuse.service: Deactivated successfully.704gitlab # [ 5.497257] systemd[1]: Finished Load Kernel Module fuse.705gitlab # [ 5.499323] systemd-journald[316]: Collecting audit messages is enabled.706gitlab # [ 5.505029] systemd[1]: Mounting FUSE Control File System...707gitlab # [ 5.508589] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...708gitlab # [ 5.511810] systemd[1]: Finished Load Kernel Modules.709gitlab # [ 5.515364] EXT4-fs (vda): re-mounted f0fd218e-56af-473f-84ce-fdcfeaf8fa93.710gitlab # [ 5.513940] systemd[1]: Queued start job for default target Multi-User System.711gitlab # [ 5.515277] systemd[1]: Unnecessary job was removed for /dev/vdb.[ 5.518056] systemd[1]: Started Journal Service.712gitlab # 713gitlab # [ 5.517580] systemd[1]: systemd-journald.service: Deactivated successfully.714gitlab # [ 5.518832] systemd-modules-load[317]: Using 3 probe threads715gitlab # [ 5.519796] systemd-modules-load[317]: Module 'atkbd' is built in716gitlab # [ 5.521394] systemd-modules-load[317]: Module 'loop' is built in717gitlab # [ 5.523312] systemd-modules-load[317]: Inserted module 'tls'718gitlab # [ 5.524354] systemd[1]: Finished Remount Root and Kernel File Systems.719gitlab # [ 5.525546] systemd[1]: Listening on Disk Image Download Service Socket.720gitlab # [ 5.527999] systemd[1]: Starting Firewall...721gitlab # [ 5.542695] systemd-oomd[318]: No swap; memory pressure usage will be degraded722gitlab # [ 5.551147] systemd[1]: Starting Flush Journal to Persistent Storage...723gitlab # [ 5.552503] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore724gitlab # [ 5.554506] systemd[1]: Starting Load/Save OS Random Seed...725gitlab # [ 5.560116] systemd[1]: Starting Apply Kernel Variables...726gitlab # [ 5.561103] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os727gitlab # [ 5.567061] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.728gitlab # [ 5.569636] systemd[1]: Mounted FUSE Control File System.729gitlab # [ 5.574884] systemd-journald[316]: Received client request to flush runtime journal.730gitlab # [ 5.592167] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.731gitlab # [ 5.593585] systemd[1]: Starting Create Static Device Nodes in /dev...732gitlab # [ 5.594694] systemd[1]: Finished Load/Save OS Random Seed.733gitlab # [ 5.595540] systemd[1]: Finished Apply Kernel Variables.734gitlab # [ 5.596475] systemd[1]: Reached target First Boot Complete.735gitlab # [ 5.597380] systemd[1]: Finished Flush Journal to Persistent Storage.736gitlab # [ 5.604352] systemd[1]: Finished Create Static Device Nodes in /dev.737gitlab # [ 5.605799] systemd[1]: Reached target Preparation for Local File Systems.738gitlab # [ 5.607889] systemd[1]: Starting Rule-based Manager for Device Events and Files...739gitlab # [ 5.675911] systemd-udevd[356]: Using default interface naming scheme 'v261'.740gitlab # [ 5.706174] systemd[1]: Started Rule-based Manager for Device Events and Files.741gitlab # [ 5.878404] systemd[1]: Finished Firewall.742gitlab # [ 6.124177] systemd[1]: Finished Coldplug All udev Devices.743gitlab # [ 6.173217] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs744gitlab # [ 6.208038] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.745gitlab # [ 6.247739] (udev-worker)[467]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.746gitlab # [ 6.251092] (udev-worker)[467]: Network interface NamePolicy= disabled on kernel command line.747gitlab # [ 6.252727] (udev-worker)[469]: Network interface NamePolicy= disabled on kernel command line.748gitlab # [ 6.255812] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped.749gitlab # [ 6.273758] systemd[1]: Condition check resulted in Virtio network device being skipped.750gitlab # [ 6.275656] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore751gitlab # [ 6.277422] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67752gitlab # [ 6.279621] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore753gitlab # [ 6.282408] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os754gitlab # [ 6.283702] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os755gitlab # [ 6.329898] mousedev: PS/2 mouse device common for all mice756gitlab # [ 6.361483] systemd[1]: Mounting /run/wrappers...757gitlab # [ 6.385254] systemd[1]: Mounted /run/wrappers.758gitlab # [ 6.386058] systemd[1]: Reached target Local File Systems.759gitlab # [ 6.386875] systemd[1]: Listening on Boot Loader Control Service Socket.760gitlab # [ 6.388652] systemd[1]: Starting register-nix-paths.service...761gitlab # [ 6.392232] systemd[1]: Starting Create SUID/SGID Wrappers...762gitlab # [ 6.400377] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.763gitlab # [ 6.403430] systemd[1]: Starting Save Transient machine-id to Disk...764gitlab # [ 6.411971] systemd[1]: Starting Create System Files and Directories...765gitlab # [ 6.427573] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.766gitlab # [ 6.428995] systemd[1]: Finished Save Transient machine-id to Disk.767gitlab # [ 6.448121] systemd[1]: Finished Create System Files and Directories.768gitlab # [ 6.453283] systemd[1]: Starting Rebuild Journal Catalog...769gitlab # [ 6.456256] systemd[1]: Starting Record System Boot/Shutdown in UTMP...770gitlab # [ 6.486733] systemd[1]: Finished Record System Boot/Shutdown in UTMP.771gitlab # [ 6.509795] systemd[1]: Finished Rebuild Journal Catalog.772gitlab # [ 6.513722] systemd[1]: Starting Update is Completed...773gitlab # [ 6.536542] systemd[1]: Finished Update is Completed.774gitlab # [ 6.562343] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.775gitlab # [ 6.564858] systemd[1]: Finished Create SUID/SGID Wrappers.776gitlab # [ 6.566351] systemd[1]: suid-sgid-wrappers.service: Consumed 123ms CPU time over 171ms wall clock time, 6.3M memory peak, 1.9M read from disk.777gitlab # [ 6.823328] systemd[1]: Finished register-nix-paths.service.778gitlab # [ 6.829678] systemd[1]: Reached target System Initialization.779gitlab # [ 6.830646] systemd[1]: Started Discard unused filesystem blocks once a week.780gitlab # [ 6.831654] systemd[1]: Started logrotate.timer.781gitlab # [ 6.832649] systemd[1]: Started Daily Cleanup of Temporary Directories.782gitlab # [ 6.833627] systemd[1]: Reached target Timer Units.783gitlab # [ 6.834364] systemd[1]: Listening on D-Bus System Message Bus Socket.784gitlab # [ 6.840250] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:09.0/virtio8/input/input3785gitlab # [ 6.840831] systemd[1]: Listening on Nix Daemon Socket.786gitlab # [ 6.881522] systemd[1]: Starting nixbot CI service socket...787gitlab # [ 6.882691] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.788gitlab # [ 6.889500] systemd[1]: Listening on nixbot CI service socket.789gitlab # [ 6.894383] systemd[1]: Reached target Socket Units.790gitlab # [ 6.898067] systemd[1]: Starting D-Bus System Message Bus...791gitlab # [ 6.907957] systemd[1]: Starting Virtual Console Setup...792gitlab # [ 6.915927] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.793gitlab # [ 6.917427] systemd[1]: Stopped Virtual Console Setup.794gitlab # [ 6.920404] systemd[1]: Starting Virtual Console Setup...795gitlab # [ 6.934022] dbus-broker-launch[564]: Looking up NSS user entry for 'systemd-timesync'...796gitlab # [ 6.935566] dbus-broker-launch[564]: NSS returned no entry for 'systemd-timesync'797gitlab # [ 6.937215] dbus-broker-launch[564]: Invalid user-name in /nix/store/9zcyylhndlnizbwn48627991zr1jxg4p-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"798gitlab # [ 6.940529] systemd[1]: Started D-Bus System Message Bus.799gitlab # [ 6.948219] dbus-broker-launch[564]: Ready800gitlab # [ 6.949243] systemd[1]: Reached target Basic System.801gitlab # [ 6.950842] systemd[1]: Started backdoor.service.802gitlab # [ 6.953108] systemd[1]: Starting gitlab-config.service...803gitlab # [ 6.958276] systemd-vconsole-setup[567]: Configuration of first virtual console was skipped, ignoring remaining ones.804gitlab # [ 6.968089] systemd[1]: Starting Import lastlog data into lastlog2 database...805gitlab # [ 6.969303] systemd[1]: Starting Logrotate configuration check...806gitlab # [ 6.974298] systemd[1]: Starting Name Service Cache Daemon (nsncd)...807gitlab # [ 6.978137] systemd[1]: Starting Post-Boot Actions...808gitlab # [ 6.984411] systemd[1]: Started Reset console on configuration changes.809gitlab # [ 6.988672] logrotate[573]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want.810gitlab # connecting to host...811gitlab # [ 6.991964] logrotate[573]: reading config file /nix/store/22afkmbkaxyx50bj9lbkkpmz7pd69hsf-logrotate.conf812gitlab # [ 6.995973] logrotate[573]: note: 'monthly' overrides previously specified 'weekly'813gitlab # [ 6.998081] logrotate[573]: Reading state from file: /var/lib/logrotate.status814gitlab # [ 6.999074] logrotate[573]: state file /var/lib/logrotate.status does not exist815gitlab # [ 7.003506] logrotate[573]: Allocating hash table for state file, size 64 entries816gitlab # [ 7.005080] logrotate[573]: Handling 4 logs817gitlab # [ 7.005688] logrotate[573]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed818gitlab: Guest shell says: b'Spawning backdoor root shell...\n'819gitlab # [ 7.007808] logrotate[573]: considering log /var/log/btmp820gitlab # [ 7.010499] logrotate[573]: Creating new state821gitlab # [ 7.011117] logrotate[573]: Now: 2026-11-13 10:33822gitlab # [ 7.012436] logrotate[573]: Last rotated at 2026-11-13 10:00823gitlab: connected to guest root shell824gitlab # [ 7.013386] logrotate[573]: log does not need rotating (log has already been rotated)825gitlab: (connecting took 7.45 seconds)826gitlab # [ 7.014386] logrotate[573]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed827gitlab: (finished: waiting for the VM to finish booting, in 7.45 seconds)828gitlab # [ 7.016677] logrotate[573]: considering log /var/log/wtmp829gitlab # [ 7.017398] logrotate[573]: Creating new state830gitlab # [ 7.017987] logrotate[573]: Now: 2026-11-13 10:33831gitlab # [ 7.018627] logrotate[573]: Last rotated at 2026-11-13 10:00832gitlab # [ 7.019371] logrotate[573]: log does not need rotating (log has already been rotated)833gitlab # [ 7.020936] logrotate[573]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed834gitlab # [ 7.022642] logrotate[573]: switching euid from 0 to 165 and egid from 0 to 165 (pid 573)835gitlab # [ 7.023825] logrotate[573]: considering log /var/gitlab/state/log/*.log836gitlab # [ 7.024824] logrotate[573]: log /var/gitlab/state/log/*.log does not exist -- skipping837gitlab # [ 7.025862] logrotate[573]: Creating new state838gitlab # [ 7.026449] logrotate[573]: switching euid from 165 to 0 and egid from 165 to 0 (pid 573)839gitlab # [ 7.027482] logrotate[573]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed840gitlab # [ 7.029265] logrotate[573]: switching euid from 0 to 60 and egid from 0 to 60 (pid 573)841gitlab # [ 7.030262] logrotate[573]: considering log /var/log/nginx/*.log842gitlab # [ 7.031296] logrotate[573]: log /var/log/nginx/*.log does not exist -- skipping843gitlab # [ 7.032994] logrotate[573]: Creating new state844gitlab # [ 7.033680] logrotate[573]: not running postrotate script, since no logs were rotated845gitlab # [ 7.035111] nsncd[574]: Nov 13 10:33:35.959 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"846gitlab # [ 7.037509] logrotate[573]: switching euid from 60 to 0 and egid from 60 to 0 (pid 573)847gitlab # [ 7.038662] systemd[1]: Starting resolvconf update...848gitlab # [ 7.041045] systemd[1]: Finished Virtual Console Setup.849gitlab # [ 7.043927] systemd[1]: Finished Logrotate configuration check.850gitlab # [ 7.045104] systemd[1]: Finished Post-Boot Actions.851gitlab # [ 7.045920] systemd[1]: Started Name Service Cache Daemon (nsncd).852gitlab # [ 7.046907] systemd[1]: Reached target Host and Network Name Lookups.853gitlab # [ 7.047936] systemd[1]: Reached target User and Group Name Lookups.854gitlab # [ 7.052397] systemd[1]: Starting User Login Management...855gitlab # [ 7.053315] systemd[1]: Finished Import lastlog data into lastlog2 database.856gitlab # [ 7.079024] systemd-logind[609]: New seat seat0.857gitlab # [ 7.081785] systemd-logind[609]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard)858gitlab # [ 7.083185] systemd[1]: Stopped target Host and Network Name Lookups.859gitlab # [ 7.085888] systemd[1]: Stopping Host and Network Name Lookups...860gitlab # [ 7.087439] systemd[1]: Stopped target User and Group Name Lookups.861gitlab # [ 7.088979] systemd[1]: Stopping User and Group Name Lookups...862gitlab # [ 7.091188] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...863gitlab # [ 7.092298] systemd-logind[609]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard)864gitlab # [ 7.093605] systemd-logind[609]: Watching system buttons on /dev/input/event0 (gpio-keys)865gitlab # [ 7.094776] systemd[1]: Started User Login Management.866gitlab # [ 7.095514] systemd[1]: nscd.service: Deactivated successfully.867gitlab # [ 7.096922] systemd[1]: Stopped Name Service Cache Daemon (nsncd).868gitlab # [ 7.097821] systemd[1]: nscd.service: Consumed 24ms CPU time over 115ms wall clock time, 7.7M memory peak, 5.8M read from disk.869gitlab # [ 7.099418] systemd[1]: Starting linger-users.service...870gitlab # [ 7.104096] systemd[1]: Starting Name Service Cache Daemon (nsncd)...871gitlab # [ 7.125241] systemd[1]: Finished resolvconf update.872gitlab # [ 7.127654] systemd[1]: Reached target Preparation for Network.873gitlab # [ 7.133162] systemd[1]: Starting DHCP Client...874gitlab # [ 7.138583] nsncd[670]: Nov 13 10:33:36.087 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"875gitlab # [ 7.143815] systemd[1]: Starting Address configuration of eth1...876gitlab # [ 7.146246] systemd[1]: Starting Extra networking commands....877gitlab # [ 7.147271] systemd[1]: Started Name Service Cache Daemon (nsncd).878gitlab # [ 7.150355] systemd[1]: linger-users.service: Deactivated successfully.879gitlab # [ 7.151529] systemd[1]: Finished linger-users.service.880gitlab # [ 7.152427] systemd[1]: Reached target Host and Network Name Lookups.881gitlab # [ 7.153414] systemd[1]: Reached target User and Group Name Lookups.882gitlab # [ 7.168935] network-addresses-eth1-start[681]: adding address 192.168.1.1/24... done883gitlab # [ 7.174175] network-addresses-eth1-start[681]: adding address 2001:db8:1::1/64... done884gitlab # [ 7.180360] systemd[1]: Finished Address configuration of eth1.885gitlab # [ 7.221783] systemd[1]: Finished Extra networking commands..886gitlab # [ 7.223471] systemd[1]: Reached target Network.887gitlab # [ 7.228231] systemd[1]: Starting gitlab-workhorse.service...888gitlab # [ 7.231299] systemd[1]: Starting Nginx Web Server...889gitlab # [ 7.235072] dhcpcd[707]: dhcpcd-10.3.2 starting890gitlab # [ 7.237596] systemd[1]: Starting PostgreSQL Server...891gitlab # [ 7.239631] dhcpcd[759]: dev: loaded udev892gitlab # [ 7.245356] systemd[1]: Starting Redis Server - redis-gitlab...893gitlab # [ 7.252302] systemd[1]: Starting Permit User Sessions...894gitlab # [ 7.258518] 8021q: 802.1Q VLAN Support v1.8895gitlab # [ 7.258942] 8021q: adding VLAN 0 to HW filter on device eth1896gitlab # [ 7.267027] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.897gitlab # [ 7.276985] systemd[1]: Finished Permit User Sessions.898gitlab # [ 7.293027] systemd[1]: Started Getty on tty1.899gitlab # [ 7.293762] systemd[1]: Reached target Login Prompts.900gitlab # [ 7.312613] cfg80211: Loading compiled-in X.509 certificates for regulatory database901gitlab # [ 7.317481] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'902gitlab # [ 7.317959] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'903gitlab # [ 7.320354] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2904gitlab # [ 7.320708] cfg80211: failed to load regulatory.db905gitlab # [ 7.359513] 8021q: adding VLAN 0 to HW filter on device eth0906gitlab # [ 7.356960] dhcpcd[759]: eth0: waiting for carrier907gitlab # [ 7.358946] dhcpcd[759]: eth0: carrier acquired908gitlab # [ 7.362884] dhcpcd[759]: DUID 00:01:00:01:32:89:a4:00:52:54:00:12:34:56909gitlab # [ 7.363963] dhcpcd[759]: eth0: IAID 00:12:34:56910gitlab # [ 7.364898] dhcpcd[759]: eth0: adding address fe80::5054:ff:fe12:3456911gitlab # [ 7.399987] nginx-pre-start[791]: nginx: the configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf syntax is ok912gitlab # [ 7.403122] nginx-pre-start[791]: nginx: configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf test is successful913gitlab # [ 7.408092] systemd[1]: Started Nginx Web Server.914gitlab # [ 7.446870] redis-server[784]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.915gitlab # [ 7.452239] redis-server[784]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo916gitlab # [ 7.453545] systemd[1]: Started Redis Server - redis-gitlab.917gitlab # [ 7.454442] redis-server[784]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=784, just started918gitlab # [ 7.456631] postgresql-pre-start[789]: The files belonging to this database system will be owned by user "postgres".919gitlab # [ 7.458187] postgresql-pre-start[789]: This user must also own the server process.920gitlab # [ 7.459360] redis-server[784]: Configuration loaded921gitlab # [ 7.460309] postgresql-pre-start[789]: The database cluster will be initialized with locale "en_US.UTF-8".922gitlab # [ 7.461649] postgresql-pre-start[789]: The default database encoding has accordingly been set to "UTF8".923gitlab # [ 7.462864] postgresql-pre-start[789]: The default text search configuration will be set to "english".924gitlab # [ 7.464275] postgresql-pre-start[789]: Data page checksums are enabled.925gitlab # [ 7.465205] postgresql-pre-start[789]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok926gitlab # [ 7.466507] postgresql-pre-start[789]: creating subdirectories ... ok927gitlab # [ 7.467367] postgresql-pre-start[789]: selecting dynamic shared memory implementation ... posix928gitlab # [ 7.468621] redis[784]: monotonic clock: ARM CNTVCT @ 1000 ticks/us929gitlab # [ 7.469783] redis[784]: Running mode=standalone, port=0.930gitlab # [ 7.470732] redis[784]: Server initialized931gitlab # [ 7.471502] redis[784]: Ready to accept connections unix932gitlab # [ 7.472496] redis[784]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.933gitlab # [ 7.506913] postgresql-pre-start[789]: selecting default "max_connections" ... 100934gitlab # [ 7.556135] postgresql-pre-start[789]: selecting default "shared_buffers" ... 128MB935gitlab # [ 7.797550] systemd[1]: Started gitlab-workhorse.service.936gitlab # [ 7.824550] dhcpcd[759]: eth0: soliciting a DHCP lease937gitlab # [ 7.836594] dhcpcd[759]: eth0: offered 10.0.2.15 from 10.0.2.2938gitlab # [ 7.856340] dhcpcd[759]: eth0: probing address 10.0.2.15/24939gitlab # [ 8.012577] postgresql-pre-start[789]: selecting default time zone ... UTC940gitlab # [ 8.014531] postgresql-pre-start[789]: creating configuration files ... ok941gitlab # [ 8.042383] gitlab-workhorse[818]: time="2026-11-13T10:33:36Z" level=info msg=Starting build_time=19700101.000000 version=19.0.4942gitlab # [ 8.045107] gitlab-workhorse[818]: time="2026-11-13T10:33:36Z" level=info msg="Using redis/go-redis"943gitlab # [ 8.047153] gitlab-workhorse[818]: time="2026-11-13T10:33:36Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix944gitlab # [ 8.254754] postgresql-pre-start[789]: running bootstrap script ... ok945gitlab # [ 8.896498] postgresql-pre-start[789]: performing post-bootstrap initialization ... ok946gitlab # [ 8.982920] dhcpcd[759]: eth0: soliciting an IPv6 router947gitlab # [ 8.984006] dhcpcd[759]: eth0: Router Advertisement from fe80::2948gitlab # [ 8.985076] dhcpcd[759]: eth0: adding address fec0::5054:ff:fe12:3456/64949gitlab # [ 8.986243] dhcpcd[759]: eth0: adding route to fec0::/64950gitlab # [ 8.987127] dhcpcd[759]: eth0: adding default route via fe80::2951gitlab # [ 9.099796] postgresql-pre-start[789]: syncing data to disk ... ok952gitlab # [ 9.101403] postgresql-pre-start[789]: initdb: warning: enabling "trust" authentication for local connections953gitlab # [ 9.103489] postgresql-pre-start[789]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.954gitlab # [ 9.107074] postgresql-pre-start[789]: Success. You can now start the database server using:955gitlab # [ 9.109024] postgresql-pre-start[789]: pg_ctl -D /var/lib/postgresql/18 -l logfile start956gitlab # [ 9.233835] postgres[843]: [843] LOG: starting PostgreSQL 18.4 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit957gitlab # [ 9.236795] postgres[843]: [843] LOG: listening on IPv6 address "::1", port 5432958gitlab # [ 9.238555] postgres[843]: [843] LOG: listening on IPv4 address "127.0.0.1", port 5432959gitlab # [ 9.240530] postgres[843]: [843] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"960gitlab # [ 9.245776] postgres[861]: [861] LOG: database system was shut down at 2026-11-13 10:33:37 GMT961gitlab # [ 9.254622] postgres[843]: [843] LOG: database system is ready to accept connections962gitlab # [ 9.260399] systemd[1]: Started PostgreSQL Server.963gitlab # [ 9.261585] systemd[1]: Starting PostgreSQL Setup Scripts...964gitlab # [ 9.497123] postgresql-setup-start[872]: CREATE DATABASE965gitlab # [ 9.530107] postgresql-setup-start[882]: CREATE ROLE966gitlab # [ 9.544621] postgresql-setup-start[884]: ALTER DATABASE967gitlab # [ 9.553281] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[877]: mkdir -p /var/gitlab/state/home/.ssh: OK968gitlab # [ 9.555468] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[877]: chmod 700 /var/gitlab/state/home/.ssh: OK969gitlab # [ 9.575034] postgresql-setup-start[893]: CREATE ROLE970gitlab # [ 9.577191] systemd[1]: Finished PostgreSQL Setup Scripts.971gitlab # [ 9.578803] systemd[1]: Reached target PostgreSQL.972gitlab # [ 9.582397] systemd[1]: Starting gitlab-postgresql.service...973gitlab # [ 9.609303] systemd[1]: Finished gitlab-config.service.974gitlab # [ 9.613041] systemd[1]: Started gitaly.service.975gitlab # [ 9.617816] systemd[1]: Starting gitlab-db-config.service...976gitlab # [ 9.725593] gitlab-postgresql-start[913]: CREATE DATABASE977gitlab # [ 9.773052] gitlab-postgresql-start[918]: CREATE EXTENSION978gitlab # [ 9.862217] gitlab-postgresql-start[920]: CREATE EXTENSION979gitlab # [ 9.866552] systemd[1]: Finished gitlab-postgresql.service.980gitlab # [ 10.252450] gitaly[905]: time="2026-11-13T10:33:39.201Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/0r0qd84zrxhqq373hk9spv8j9pg0qlnq-gitaly.toml' and fix the errors that are printed." pid=905981gitlab # [ 10.268192] gitaly[905]: time="2026-11-13T10:33:39.201Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=905982gitlab # [ 10.270067] gitaly[905]: time="2026-11-13T10:33:39.201Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=905983gitlab # [ 10.275008] gitaly[905]: time="2026-11-13T10:33:39.201Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=905984gitlab # [ 10.279237] gitaly[905]: time="2026-11-13T10:33:39.201Z" level=info msg="Starting Gitaly" pid=905 version=985gitlab # [ 10.449486] gitaly[905]: time="2026-11-13T10:33:39.397Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=195 pid=905986gitlab # [ 10.451574] gitaly[905]: time="2026-11-13T10:33:39.397Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=905987gitlab # [ 10.453768] gitaly[905]: time="2026-11-13T10:33:39.398Z" level=info msg="finished initializing command factory" duration_ms=0 pid=905988gitlab # [ 10.455320] gitaly[905]: time="2026-11-13T10:33:39.398Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-905/git-exec-1262491896.d/git pid=905989gitlab # [ 10.457564] gitaly[905]: time="2026-11-13T10:33:39.403Z" level=info msg="finished detecting git version" duration_ms=4 pid=905990gitlab # [ 10.459508] gitaly[905]: time="2026-11-13T10:33:39.403Z" level=info msg="using Git version" pid=905 version=2.53.g991gitlab # [ 10.461337] gitaly[905]: time="2026-11-13T10:33:39.403Z" level=info msg="clearing disk cache object folder" pid=905 storage=default992gitlab # [ 10.463383] gitaly[905]: time="2026-11-13T10:33:39.403Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache193350148 pid=905 storage=default993gitlab # [ 10.466726] gitaly[905]: time="2026-11-13T10:33:39.403Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache193350148 pid=905 storage=default994gitlab # [ 10.470532] gitaly[905]: time="2026-11-13T10:33:39.403Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=905995gitlab # [ 10.474337] gitaly[905]: time="2026-11-13T10:33:39.403Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=905996gitlab # [ 10.477492] gitaly[905]: time="2026-11-13T10:33:39.403Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache193350148 clear_duration_ms=0 pid=905 storage=default997gitlab # [ 10.480956] gitaly[905]: time="2026-11-13T10:33:39.417Z" level=info msg="counting repositories" pid=905998gitlab # [ 10.482606] gitaly[905]: time="2026-11-13T10:33:39.417Z" level=info msg="starting to count repositories" pid=905 storage_path=/var/gitlab/state/repositories999gitlab # [ 10.485050] gitaly[905]: time="2026-11-13T10:33:39.417Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=905 storage_path=/var/gitlab/state/repositories1000gitlab # [ 10.487756] gitaly[905]: time="2026-11-13T10:33:39.417Z" level=info msg="completed counting all repositories" counting_duration=0 pid=9051001gitlab # [ 10.489947] gitaly[905]: time="2026-11-13T10:33:39.417Z" level=info msg="finished tempdir cleaner walk" pid=905 storage=default time_ms=01002gitlab # [ 10.492109] gitaly[905]: time="2026-11-13T10:33:39.417Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=9051003gitlab # [ 10.494699] gitaly[905]: time="2026-11-13T10:33:39.417Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=905 schema=unix1004gitlab # [ 10.497259] gitaly[905]: time="2026-11-13T10:33:39.418Z" level=info msg="listening at address" address=/run/gitaly/gitaly-905/sock.d/intern pid=905 schema=unix1005gitlab # [ 10.499551] gitaly[905]: time="2026-11-13T10:33:39.425Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=9051006gitlab # [ 10.501703] gitaly[905]: time="2026-11-13T10:33:39.425Z" level=info msg="Started Gitaly" duration_ms=223 pid=9051007gitlab # [ 10.503468] gitaly[905]: time="2026-11-13T10:33:39.425Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=9051008gitlab # [ 10.505633] gitaly[905]: time="2026-11-13T10:33:39.425Z" level=info msg="maintenance: daily scheduled" pid=905 scheduled="2026-11-13 12:00:00 +0000 UTC"1009gitlab # [ 10.682889] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[912]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1010gitlab # [ 10.687782] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[912]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1011gitlab # [ 10.696209] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[912]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1012gitlab # [ 12.432302] dhcpcd[759]: eth0: leased 10.0.2.15 for 86400 seconds1013gitlab # [ 12.433836] dhcpcd[759]: eth0: adding route to 10.0.2.0/241014gitlab # [ 12.435161] dhcpcd[759]: eth0: adding default route via 10.0.2.21015gitlab # [ 12.534176] systemd[1]: Started DHCP Client.1016gitlab # [ 12.536251] systemd[1]: Reached target Network is Online.1017gitlab # [ 13.936072] gitaly[905]: time="2026-11-13T10:33:42.884Z" level=info msg="License database preloaded" duration_ms=3683 pid=9051018gitlab # [ 56.822410] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[912]: W, [2026-11-13T10:34:25.770706 #912] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1019gitlab # [ 58.122493] gitlab-workhorse[818]: time="2026-11-13T10:34:27Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1020gitlab # [ 59.870933] postgres[1452]: [1452] ERROR: relation "feature_gates" does not exist at character 5231021gitlab # [ 59.873133] postgres[1452]: [1452] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod),1022gitlab # [ 59.875085] postgres[1452]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1023gitlab # [ 59.877215] postgres[1452]: c.collname, col_description(a.attrelid, a.attnum) AS comment,1024gitlab # [ 59.879056] postgres[1452]: attidentity AS identity,1025gitlab # [ 59.880401] postgres[1452]: attgenerated as attgenerated1026gitlab # [ 59.881644] postgres[1452]: FROM pg_attribute a1027gitlab # [ 59.882664] postgres[1452]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1028gitlab # [ 59.884640] postgres[1452]: LEFT JOIN pg_type t ON a.atttypid = t.oid1029gitlab # [ 59.886069] postgres[1452]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1030gitlab # [ 59.888281] postgres[1452]: WHERE a.attrelid = '"feature_gates"'::regclass1031gitlab # [ 59.889790] postgres[1452]: AND a.attnum > 0 AND NOT a.attisdropped1032gitlab # [ 59.891197] postgres[1452]: ORDER BY a.attnum1033gitlab # [ 59.892229] postgres[1452]:1034gitlab # [ 64.988494] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1503]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping1035gitlab # [ 64.988782] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1503]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping1036gitlab # [ 68.129261] gitlab-workhorse[818]: time="2026-11-13T10:34:37Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1037gitlab # [ 78.137406] gitlab-workhorse[818]: time="2026-11-13T10:34:47Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1038gitlab # [ 87.153357] postgres[1606]: [1606] ERROR: relation "table_without_models" does not exist1039gitlab # [ 87.153638] postgres[1606]: [1606] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1040gitlab # [ 88.145520] gitlab-workhorse[818]: time="2026-11-13T10:34:57Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1041gitlab # [ 98.155324] gitlab-workhorse[818]: time="2026-11-13T10:35:07Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1042gitlab # [ 104.244742] systemd[1]: Finished gitlab-db-config.service.1043gitlab # [ 104.251146] systemd[1]: Started gitlab-sidekiq.service.1044gitlab # [ 104.257670] systemd[1]: Starting gitlab.service...1045gitlab # [ 104.926668] bundle[1845]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1046gitlab # [ 104.926957] bundle[1845]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1047gitlab # [ 104.928824] bundle[1845]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1048gitlab # [ 108.161344] gitlab-workhorse[818]: time="2026-11-13T10:35:17Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1049gitlab # [ 109.606080] sidekiq-cluster[1844]: {"severity":"INFO","time":"2026-11-13T10:35:18.527Z","message":"Starting cluster with 1 processes"}1050gitlab # [ 109.734529] bundle[1845]: {"timestamp":"2026-11-13T10:35:18.652Z","pid":1845,"message":"* Enabling systemd notification integration"}1051gitlab # [ 109.737220] bundle[1845]: {"timestamp":"2026-11-13T10:35:18.683Z","pid":1845,"message":"Puma starting in cluster mode..."}1052gitlab # [ 109.739521] bundle[1845]: {"timestamp":"2026-11-13T10:35:18.683Z","pid":1845,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"}1053gitlab # [ 109.742121] bundle[1845]: {"timestamp":"2026-11-13T10:35:18.683Z","pid":1845,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1054gitlab # [ 109.745216] bundle[1845]: {"timestamp":"2026-11-13T10:35:18.683Z","pid":1845,"message":"* Min threads: 0"}1055gitlab # [ 109.747505] bundle[1845]: {"timestamp":"2026-11-13T10:35:18.683Z","pid":1845,"message":"* Max threads: 4"}1056gitlab # [ 109.749735] bundle[1845]: {"timestamp":"2026-11-13T10:35:18.683Z","pid":1845,"message":"* Environment: production"}1057gitlab # [ 109.751954] bundle[1845]: {"timestamp":"2026-11-13T10:35:18.683Z","pid":1845,"message":"* Master PID: 1845"}1058gitlab # [ 109.754140] bundle[1845]: {"timestamp":"2026-11-13T10:35:18.683Z","pid":1845,"message":"* Workers: 2"}1059gitlab # [ 109.756204] bundle[1845]: {"timestamp":"2026-11-13T10:35:18.683Z","pid":1845,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"}1060gitlab # [ 109.758760] bundle[1845]: {"timestamp":"2026-11-13T10:35:18.683Z","pid":1845,"message":"* Preloading application"}1061gitlab # [ 118.170972] gitlab-workhorse[818]: time="2026-11-13T10:35:27Z" level=error correlation_id= duration_ms=1 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1062gitlab # [ 128.171786] gitlab-workhorse[818]: time="2026-11-13T10:35:37Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1063gitlab # [ 138.172803] gitlab-workhorse[818]: time="2026-11-13T10:35:47Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1064gitlab # [ 148.181163] gitlab-workhorse[818]: time="2026-11-13T10:35:57Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1065gitlab # [ 148.655991] bundle[1845]: W, [2026-11-13T10:35:57.604564 #1845] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1066gitlab # [ 152.092632] sidekiq-cluster[1894]: 2026-11-13T10:36:01.041Z pid=1894 tid=wr2 INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>1894, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>425775104}1067gitlab # [ 152.411586] sidekiq-cluster[1894]: W, [2026-11-13T10:36:01.360586 #1894] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1068gitlab # [ 152.559896] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:01.508Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1069gitlab # [ 155.853899] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.802Z","message":"GitLab reliable fetch activated!"}1070gitlab # [ 155.854136] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.802Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1071gitlab # [ 155.890073] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.839Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1072gitlab # [ 155.893943] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.841Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"}1073gitlab # [ 155.897729] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.845Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1074gitlab # [ 155.901231] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.848Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1075gitlab # [ 155.905131] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.852Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1076gitlab # [ 155.908620] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.855Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1077gitlab # [ 155.912173] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.858Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1078gitlab # [ 155.915371] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.860Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1079gitlab # [ 155.918813] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.863Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1080gitlab # [ 155.922269] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.865Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1081gitlab # [ 155.925963] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.868Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1082gitlab # [ 155.929340] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.871Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1083gitlab # [ 155.932305] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.873Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1084gitlab # [ 155.935340] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.876Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1085gitlab # [ 155.938409] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.878Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1086gitlab # [ 155.941351] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.880Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1087gitlab # [ 155.943980] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.884Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1088gitlab # [ 155.946620] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.887Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1089gitlab # [ 155.949433] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.891Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1090gitlab # [ 155.951941] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.894Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1091gitlab # [ 155.954811] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.897Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1092gitlab # [ 155.957629] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.900Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1093gitlab # [ 155.959963] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.902Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1094gitlab # [ 155.962456] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.905Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1095gitlab # [ 155.964918] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.907Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1096gitlab # [ 155.967127] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.910Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1097gitlab # [ 155.969388] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.913Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1098gitlab # [ 155.972497] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.916Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1099gitlab # [ 155.974665] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.919Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1100gitlab # [ 155.976994] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.922Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1101gitlab # [ 155.979067] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.924Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1102gitlab # [ 155.981494] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.926Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1103gitlab # [ 155.983579] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.929Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1104gitlab # [ 155.985970] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.932Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1105gitlab # [ 155.988193] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.934Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1106gitlab # [ 155.990377] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.936Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1107gitlab # [ 155.992650] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.938Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1108gitlab # [ 155.994747] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.941Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1109gitlab # [ 155.996982] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.943Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1110gitlab # [ 155.999031] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.945Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1111gitlab # [ 156.001323] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.948Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1112gitlab # [ 156.003468] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.950Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1113gitlab # [ 156.005814] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.952Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1114gitlab # [ 156.007945] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.954Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1115gitlab # [ 156.010361] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.957Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1116gitlab # [ 156.012762] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.959Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1117gitlab # [ 156.014960] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.961Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1118gitlab # [ 156.017257] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.963Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1119gitlab # [ 156.019392] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.966Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1120gitlab # [ 156.021725] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.968Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1121gitlab # [ 156.023893] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.970Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1122gitlab # [ 156.026187] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.973Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1123gitlab # [ 156.039638] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.988Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1124gitlab # [ 156.042717] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.991Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1125gitlab # [ 156.045330] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.994Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1126gitlab # [ 156.048093] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.997Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1127gitlab # [ 156.050547] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:04.999Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1128gitlab # [ 156.053515] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.002Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1129gitlab # [ 156.055773] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.004Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1130gitlab # [ 156.058222] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.007Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1131gitlab # [ 156.134585] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.082Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1132gitlab # [ 156.138025] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.086Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1133gitlab # [ 156.141522] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.088Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1134gitlab # [ 156.145061] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.091Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1135gitlab # [ 156.148709] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.095Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1136gitlab # [ 156.152172] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.097Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1137gitlab # [ 156.155427] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.100Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1138gitlab # [ 156.159062] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.102Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1139gitlab # [ 156.162902] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.105Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1140gitlab # [ 156.166719] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.106Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1141gitlab # [ 156.170429] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.109Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1142gitlab # [ 156.174938] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.111Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1143gitlab # [ 156.178693] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.113Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1144gitlab # [ 156.182585] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.116Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1145gitlab # [ 156.186416] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.117Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1146gitlab # [ 156.190273] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.120Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1147gitlab # [ 156.193897] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.122Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1148gitlab # [ 156.197784] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.125Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1149gitlab # [ 156.201067] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.126Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1150gitlab # [ 156.204133] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.129Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1151gitlab # [ 156.207349] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.131Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1152gitlab # [ 156.210738] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.133Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1153gitlab # [ 156.213837] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.136Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1154gitlab # [ 156.216666] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.138Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1155gitlab # [ 156.219574] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.140Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1156gitlab # [ 156.222368] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.142Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1157gitlab # [ 156.225195] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.145Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1158gitlab # [ 156.227827] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.147Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1159gitlab # [ 156.230564] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.149Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1160gitlab # [ 156.233142] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.151Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1161gitlab # [ 156.235655] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.153Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1162gitlab # [ 156.238151] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.155Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1163gitlab # [ 156.240577] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.157Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1164gitlab # [ 156.243114] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.159Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1165gitlab # [ 156.245254] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.161Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1166gitlab # [ 156.247607] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.165Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1167gitlab # [ 156.250005] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.166Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1168gitlab # [ 156.252336] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.169Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1169gitlab # [ 156.254539] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.180Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1170gitlab # [ 156.256824] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.182Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1171gitlab # [ 156.259032] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.184Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1172gitlab # [ 156.261454] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.187Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1173gitlab # [ 156.263548] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.191Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1174gitlab # [ 156.265912] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.194Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1175gitlab # [ 156.268307] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.196Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1176gitlab # [ 156.270665] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.198Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1177gitlab # [ 156.272865] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.200Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1178gitlab # [ 156.275141] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.204Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1179gitlab # [ 156.277487] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:05.205Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1180gitlab # [ 158.181924] gitlab-workhorse[818]: time="2026-11-13T10:36:07Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1181gitlab # [ 168.182639] gitlab-workhorse[818]: time="2026-11-13T10:36:17Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1182gitlab # [ 178.189556] gitlab-workhorse[818]: time="2026-11-13T10:36:27Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1183gitlab # [ 188.197506] gitlab-workhorse[818]: time="2026-11-13T10:36:37Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1184gitlab # [ 192.740227] postgres[2641]: [2641] ERROR: relation "table_without_models" does not exist1185gitlab # [ 192.740474] postgres[2641]: [2641] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1186gitlab # [ 197.636866] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:46.585Z","message":"Booted Rails 7.2.3.1 application in production environment"}1187gitlab # [ 197.644144] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:46.592Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1188gitlab # [ 197.647442] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:46.593Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1189gitlab # [ 197.650436] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:46.593Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1190gitlab # [ 197.653731] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:46.595Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1191gitlab # [ 197.659931] postgres[2688]: [2688] ERROR: relation "table_without_models" does not exist1192gitlab # [ 197.661819] postgres[2688]: [2688] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1193gitlab # [ 198.205444] gitlab-workhorse[818]: time="2026-11-13T10:36:47Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1194gitlab # [ 200.368516] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:49.310Z","message":"Cleaning working queues"}1195gitlab # [ 200.368795] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:49.312Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"7356ab45157a8689e2fdb02a","created_at":"2026-11-13T10:35:12.463Z","trace_propagation_headers":{"sentry-trace":"8dab754422044c05b3d201760215091d-f42240a5362d4d95","baggage":"sentry-trace_id=8dab754422044c05b3d201760215091d,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"de96d96b1c711de9845583e05259c641","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-11-13T10:35:12.489Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-7356ab45157a8689e2fdb02a: start","job_status":"start","queue_duration_s":96.82319,"scheduling_latency_s":96.82325}1196gitlab # [ 200.594788] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:36:49.543Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"7356ab45157a8689e2fdb02a","created_at":"2026-11-13T10:35:12.463Z","trace_propagation_headers":{"sentry-trace":"8dab754422044c05b3d201760215091d-f42240a5362d4d95","baggage":"sentry-trace_id=8dab754422044c05b3d201760215091d,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"de96d96b1c711de9845583e05259c641","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-11-13T10:35:12.489Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-7356ab45157a8689e2fdb02a: done: 0.231107 sec","job_status":"done","queue_duration_s":96.82319,"scheduling_latency_s":96.82325,"redis_calls":29,"redis_duration_s":0.006911,"redis_read_bytes":282,"redis_write_bytes":3632,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.004056,"redis_feature_flag_read_bytes":272,"redis_feature_flag_write_bytes":2385,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001779,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001076,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":15,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.120232,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.12344636,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.231107,"completed_at":"2026-11-13T10:36:49.543Z","load_balancing_strategy":"primary","db_duration_s":0.059485,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1197gitlab # [ 204.435871] bundle[1845]: {"timestamp":"2026-11-13T10:36:53.384Z","pid":1845,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1198gitlab # [ 204.439129] bundle[1845]: {"timestamp":"2026-11-13T10:36:53.384Z","pid":1845,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1199gitlab # [ 204.441945] bundle[1845]: {"timestamp":"2026-11-13T10:36:53.384Z","pid":1845,"message":"! #\u003cThread:0x0000e90738d1b108 /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1200gitlab # [ 204.450285] bundle[1845]: {"timestamp":"2026-11-13T10:36:53.384Z","pid":1845,"message":"! #\u003cThread:0x0000e907385a4fc8 /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1201gitlab # [ 204.458685] bundle[1845]: {"timestamp":"2026-11-13T10:36:53.384Z","pid":1845,"message":"Use Ctrl-C to stop"}1202gitlab # [ 204.460754] bundle[1845]: {"timestamp":"2026-11-13T10:36:53.385Z","pid":1845,"message":"Sending status to systemd every 1.0 sec"}1203gitlab # [ 205.138856] bundle[1845]: {"timestamp":"2026-11-13T10:36:54.087Z","pid":1845,"message":"- Worker 0 (PID: 2787) booted in 0.67s, phase: 0"}1204gitlab # [ 205.170550] bundle[1845]: {"timestamp":"2026-11-13T10:36:54.119Z","pid":1845,"message":"- Worker 1 (PID: 2791) booted in 0.68s, phase: 0"}1205gitlab # [ 205.177246] systemd[1]: Started gitlab.service.1206gitlab # [ 205.178391] systemd[1]: Reached target Common target for all GitLab services..1207gitlab # [ 205.180478] systemd[1]: Starting setup-gitlab.service...1208gitlab # [ 205.209946] setup-gitlab-start[2819]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1209gitlab # [ 209.548977] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:36:58 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 43071210gitlab # [ 209.554940] setup-gitlab-start[2819]: + TOKEN=nixbot-test-token-of-20-chars1211gitlab # [ 209.556804] setup-gitlab-start[2819]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n'1212gitlab # [ 209.571173] runuser[2850]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1213gitlab # [ 210.703354] setup-gitlab-start[2851]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1214gitlab # [ 210.703602] setup-gitlab-start[2851]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1215gitlab # [ 210.705557] setup-gitlab-start[2851]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1216gitlab # [ 211.562235] setup-gitlab-start[2851]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1217gitlab # [ 211.562548] setup-gitlab-start[2851]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1218gitlab # [ 211.565196] setup-gitlab-start[2851]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1219gitlab # [ 218.763565] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:07.711Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"ed97f143da532e17abebb933","created_at":"2026-11-13T10:37:07.631Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f0ba95bd6958a11c772db919b7a226fc","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:07.709Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-ed97f143da532e17abebb933: start","job_status":"start","queue_duration_s":0.002164,"scheduling_latency_s":0.002208}1220gitlab # [ 218.907545] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:07.855Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"ed97f143da532e17abebb933","created_at":"2026-11-13T10:37:07.631Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f0ba95bd6958a11c772db919b7a226fc","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:07.709Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-ed97f143da532e17abebb933: done: 0.143995 sec","job_status":"done","queue_duration_s":0.002164,"scheduling_latency_s":0.002208,"redis_calls":13,"redis_duration_s":0.01365,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001402,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.008297,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003951,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040476,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01962704,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.143995,"completed_at":"2026-11-13T10:37:07.855Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.020858,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1221gitlab # [ 218.975828] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:07.862Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"4d9138aa97a56cc9f35a2daf","created_at":"2026-11-13T10:37:07.730Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dad53d5ccd979304ded7423d60990498","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:07.748Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-4d9138aa97a56cc9f35a2daf: start","job_status":"start","queue_duration_s":0.113756,"scheduling_latency_s":0.113793}1222gitlab # [ 219.141188] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:08.089Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"4d9138aa97a56cc9f35a2daf","created_at":"2026-11-13T10:37:07.730Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dad53d5ccd979304ded7423d60990498","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:07.748Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-4d9138aa97a56cc9f35a2daf: done: 0.227581 sec","job_status":"done","queue_duration_s":0.113756,"scheduling_latency_s":0.113793,"redis_calls":14,"redis_duration_s":0.032492,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002139,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.007436,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.022917,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.044,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.043,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.079548,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00080462,"exclusive_lock_hold_duration_s":0.1570688,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.227581,"completed_at":"2026-11-13T10:37:08.089Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.054078,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1223gitlab # [ 219.210749] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:08.097Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"8092ee87e9054b3a9d0f99d9","created_at":"2026-11-13T10:37:07.759Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"caad6081a3af16af68f024921680c8ac","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:07.766Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-8092ee87e9054b3a9d0f99d9: start","job_status":"start","queue_duration_s":0.330901,"scheduling_latency_s":0.330934}1224gitlab # [ 219.227834] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:08.154Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"8092ee87e9054b3a9d0f99d9","created_at":"2026-11-13T10:37:07.759Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"caad6081a3af16af68f024921680c8ac","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:07.766Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-8092ee87e9054b3a9d0f99d9: done: 0.057209 sec","job_status":"done","queue_duration_s":0.330901,"scheduling_latency_s":0.330934,"redis_calls":12,"redis_duration_s":0.014741,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001547,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.009686,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003508,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023168,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00159316,"exclusive_lock_hold_duration_s":0.01002,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.057209,"completed_at":"2026-11-13T10:37:08.154Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004556,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1225gitlab # [ 219.273145] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:08.158Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"05d22622abaaed6176866eac","created_at":"2026-11-13T10:37:07.771Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7a1d9ab74dac5442ae49e96521a71ccc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:07.781Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-05d22622abaaed6176866eac: start","job_status":"start","queue_duration_s":0.37691,"scheduling_latency_s":0.376943}1226gitlab # [ 219.288994] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:08.219Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"05d22622abaaed6176866eac","created_at":"2026-11-13T10:37:07.771Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7a1d9ab74dac5442ae49e96521a71ccc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:07.781Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-05d22622abaaed6176866eac: done: 0.060803 sec","job_status":"done","queue_duration_s":0.37691,"scheduling_latency_s":0.376943,"redis_calls":12,"redis_duration_s":0.01931,"redis_read_bytes":11,"redis_write_bytes":1482,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.006357,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.011096,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001857,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022199,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00394766,"duration_s":0.060803,"completed_at":"2026-11-13T10:37:08.219Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003879,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1227gitlab # [ 219.332590] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:08.224Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"22be0e873926b4a273418a20","created_at":"2026-11-13T10:37:07.805Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"50e1ffd099c086891dd87afaf3eb29ae","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:07.818Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-22be0e873926b4a273418a20: start","job_status":"start","queue_duration_s":0.406215,"scheduling_latency_s":0.406251}1228gitlab # [ 219.406520] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:08.355Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"22be0e873926b4a273418a20","created_at":"2026-11-13T10:37:07.805Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"50e1ffd099c086891dd87afaf3eb29ae","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:07.818Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-22be0e873926b4a273418a20: done: 0.130652 sec","job_status":"done","queue_duration_s":0.406215,"scheduling_latency_s":0.406251,"redis_calls":16,"redis_duration_s":0.039394,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.004501,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.033767,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001126,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038208,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00035344,"duration_s":0.130652,"completed_at":"2026-11-13T10:37:08.354Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.030996,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1229gitlab # [ 219.469025] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:08.357Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"40bb2dd6d4e0371b9aee47f2","created_at":"2026-11-13T10:37:07.846Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"73b4e132d01cda7a23981c390f98fa21","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:07.858Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-40bb2dd6d4e0371b9aee47f2: start","job_status":"start","queue_duration_s":0.49831,"scheduling_latency_s":0.498343}1230gitlab # [ 219.485106] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:08.422Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"40bb2dd6d4e0371b9aee47f2","created_at":"2026-11-13T10:37:07.846Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"73b4e132d01cda7a23981c390f98fa21","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:07.858Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-40bb2dd6d4e0371b9aee47f2: done: 0.065121 sec","job_status":"done","queue_duration_s":0.49831,"scheduling_latency_s":0.498343,"redis_calls":12,"redis_duration_s":0.003345,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001246,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001163,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000936,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025636,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00080234,"exclusive_lock_hold_duration_s":0.01824542,"duration_s":0.065121,"completed_at":"2026-11-13T10:37:08.422Z","load_balancing_strategy":"primary","db_duration_s":0.022885,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1231gitlab # [ 219.529872] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:08.424Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"05b603d0c1acdd9be11af285","created_at":"2026-11-13T10:37:07.868Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fbe15d791f1e786ef9f9f9e168907249","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:07.905Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-05b603d0c1acdd9be11af285: start","job_status":"start","queue_duration_s":0.518987,"scheduling_latency_s":0.51902}1232gitlab # [ 219.673024] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:08.621Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"05b603d0c1acdd9be11af285","created_at":"2026-11-13T10:37:07.868Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fbe15d791f1e786ef9f9f9e168907249","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:07.905Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-05b603d0c1acdd9be11af285: done: 0.197001 sec","job_status":"done","queue_duration_s":0.518987,"scheduling_latency_s":0.51902,"redis_calls":10,"redis_duration_s":0.0024980000000000002,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000506,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001472,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00052,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026448,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0005258,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.197001,"completed_at":"2026-11-13T10:37:08.621Z","load_balancing_strategy":"primary","db_duration_s":0.150771,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1233gitlab # [ 219.739710] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:08.623Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"e4037111c749f1397b0ad232","created_at":"2026-11-13T10:37:07.957Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6bb2c2dea92ed69c9aff6dd08d4aa903","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:07.974Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-e4037111c749f1397b0ad232: start","job_status":"start","queue_duration_s":0.649518,"scheduling_latency_s":0.649548}1234gitlab # [ 220.047894] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:08.995Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"e4037111c749f1397b0ad232","created_at":"2026-11-13T10:37:07.957Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6bb2c2dea92ed69c9aff6dd08d4aa903","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:07.974Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-e4037111c749f1397b0ad232: done: 0.37212 sec","job_status":"done","queue_duration_s":0.649518,"scheduling_latency_s":0.649548,"redis_calls":565,"redis_duration_s":0.07635600000000001,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000456,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00123,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.07467,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.333679,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003614,"duration_s":0.37212,"completed_at":"2026-11-13T10:37:08.995Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000598,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1235gitlab # [ 220.114059] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:08.998Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"e6ed4e8bea8c248a464372c1","created_at":"2026-11-13T10:37:07.980Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c1b17531aa1532b62c899a7797ecb342","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:07.994Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-e6ed4e8bea8c248a464372c1: start","job_status":"start","queue_duration_s":1.003283,"scheduling_latency_s":1.003312}1236gitlab # [ 220.132187] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.013Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1237gitlab # [ 220.136535] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.025Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"e6ed4e8bea8c248a464372c1","created_at":"2026-11-13T10:37:07.980Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c1b17531aa1532b62c899a7797ecb342","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:07.994Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-e6ed4e8bea8c248a464372c1: done: 0.027787 sec","job_status":"done","queue_duration_s":1.003283,"scheduling_latency_s":1.003312,"redis_calls":10,"redis_duration_s":0.002702,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000599,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001623,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00048,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01618,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00079514,"duration_s":0.027787,"completed_at":"2026-11-13T10:37:09.025Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000927,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1238gitlab # [ 220.184800] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.027Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"3cb5eea85aa1613d96df779e","created_at":"2026-11-13T10:37:08.030Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2e6a9a02be89231b8019b574b068b5f4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-13T10:37:08.039Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-3cb5eea85aa1613d96df779e: start","job_status":"start","queue_duration_s":0.987801,"scheduling_latency_s":0.987831}1239gitlab # [ 220.198647] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.075Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"3cb5eea85aa1613d96df779e","created_at":"2026-11-13T10:37:08.030Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2e6a9a02be89231b8019b574b068b5f4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-13T10:37:08.039Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-3cb5eea85aa1613d96df779e: done: 0.047756 sec","job_status":"done","queue_duration_s":0.987801,"scheduling_latency_s":0.987831,"redis_calls":10,"redis_duration_s":0.002456,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000595,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001356,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000505,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032529,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00035764,"duration_s":0.047756,"completed_at":"2026-11-13T10:37:09.075Z","load_balancing_strategy":"primary","db_duration_s":0.004374,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1240gitlab # [ 220.240993] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.078Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"8ac205a6ad3ca751b3320ac5","created_at":"2026-11-13T10:37:08.071Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4c5f8540df7ec32f96517f42e83658c5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-13T10:37:08.086Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-8ac205a6ad3ca751b3320ac5: start","job_status":"start","queue_duration_s":0.991992,"scheduling_latency_s":0.992028}1241gitlab # [ 220.254748] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.095Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1242gitlab # [ 220.258178] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.107Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"8ac205a6ad3ca751b3320ac5","created_at":"2026-11-13T10:37:08.071Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4c5f8540df7ec32f96517f42e83658c5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-13T10:37:08.086Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-8ac205a6ad3ca751b3320ac5: done: 0.029311 sec","job_status":"done","queue_duration_s":0.991992,"scheduling_latency_s":0.992028,"redis_calls":10,"redis_duration_s":0.001958,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000417,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001131,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00041,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016336,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00031168,"duration_s":0.029311,"completed_at":"2026-11-13T10:37:09.107Z","load_balancing_strategy":"primary","db_duration_s":0.000479,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1243gitlab # [ 220.300734] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.109Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"c48514d8752261c1594ecfdb","created_at":"2026-11-13T10:37:08.096Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"11db46fe885cc708b1e6891f00123615","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:08.115Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-c48514d8752261c1594ecfdb: start","job_status":"start","queue_duration_s":0.99341,"scheduling_latency_s":0.993435}1244gitlab # [ 220.314613] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.156Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"c48514d8752261c1594ecfdb","created_at":"2026-11-13T10:37:08.096Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"11db46fe885cc708b1e6891f00123615","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:08.115Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-c48514d8752261c1594ecfdb: done: 0.046907 sec","job_status":"done","queue_duration_s":0.99341,"scheduling_latency_s":0.993435,"redis_calls":19,"redis_duration_s":0.004071,"redis_read_bytes":22,"redis_write_bytes":4781,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000989,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000277,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3011,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.002138,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000667,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033521,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00031268,"duration_s":0.046907,"completed_at":"2026-11-13T10:37:09.156Z","load_balancing_strategy":"primary","db_duration_s":0.002447,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1245gitlab # [ 220.359229] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.158Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"bab8e3c4e43a3e3025374cd8","created_at":"2026-11-13T10:37:08.136Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3d2bf2d28c2abbe6778a67f7adf8aefd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:08.151Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-bab8e3c4e43a3e3025374cd8: start","job_status":"start","queue_duration_s":1.006592,"scheduling_latency_s":1.006614}1246gitlab # [ 220.374542] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.220Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"bab8e3c4e43a3e3025374cd8","created_at":"2026-11-13T10:37:08.136Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3d2bf2d28c2abbe6778a67f7adf8aefd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:08.151Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-bab8e3c4e43a3e3025374cd8: done: 0.061772 sec","job_status":"done","queue_duration_s":1.006592,"scheduling_latency_s":1.006614,"redis_calls":12,"redis_duration_s":0.002551,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000347,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001137,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001067,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037287,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00063294,"exclusive_lock_hold_duration_s":0.03159558,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.061772,"completed_at":"2026-11-13T10:37:09.220Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015253,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1247gitlab # [ 220.420743] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.222Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"16baf8a6c18245340946e7b7","created_at":"2026-11-13T10:37:08.162Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1d6e829ad3f393e4d7ea50655a74cf27","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-13T10:37:08.179Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-16baf8a6c18245340946e7b7: start","job_status":"start","queue_duration_s":1.043214,"scheduling_latency_s":1.043236}1248gitlab # [ 220.434524] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.236Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1249gitlab # [ 220.438460] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.245Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"16baf8a6c18245340946e7b7","created_at":"2026-11-13T10:37:08.162Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1d6e829ad3f393e4d7ea50655a74cf27","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-13T10:37:08.179Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-16baf8a6c18245340946e7b7: done: 0.023191 sec","job_status":"done","queue_duration_s":1.043214,"scheduling_latency_s":1.043236,"redis_calls":10,"redis_duration_s":0.0018830000000000001,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000399,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001098,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000386,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012581,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00028248,"duration_s":0.023191,"completed_at":"2026-11-13T10:37:09.245Z","load_balancing_strategy":"primary","db_duration_s":0.000467,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1250gitlab # [ 220.480899] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.246Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"e3f7516dc57440b2b1986556","created_at":"2026-11-13T10:37:08.194Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"986602f65519bf3cc53887aca2c473c7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:08.208Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-e3f7516dc57440b2b1986556: start","job_status":"start","queue_duration_s":1.037901,"scheduling_latency_s":1.037918}1251gitlab # [ 220.495887] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.259Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1252gitlab # [ 220.499560] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.268Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"e3f7516dc57440b2b1986556","created_at":"2026-11-13T10:37:08.194Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"986602f65519bf3cc53887aca2c473c7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:08.208Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-e3f7516dc57440b2b1986556: done: 0.021734 sec","job_status":"done","queue_duration_s":1.037901,"scheduling_latency_s":1.037918,"redis_calls":10,"redis_duration_s":0.001786,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000357,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001033,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000396,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01239,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00027532,"duration_s":0.021734,"completed_at":"2026-11-13T10:37:09.268Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000367,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1253gitlab # [ 220.543505] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.269Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"1f913532ef723ca59f0e0cae","created_at":"2026-11-13T10:37:08.225Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"24d0aeb26a6ebe23d7e99134ca10e695","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:08.233Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-1f913532ef723ca59f0e0cae: start","job_status":"start","queue_duration_s":1.036428,"scheduling_latency_s":1.036446}1254gitlab # [ 220.559638] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.292Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"1f913532ef723ca59f0e0cae","created_at":"2026-11-13T10:37:08.225Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"24d0aeb26a6ebe23d7e99134ca10e695","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:08.233Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-1f913532ef723ca59f0e0cae: done: 0.022636 sec","job_status":"done","queue_duration_s":1.036428,"scheduling_latency_s":1.036446,"redis_calls":10,"redis_duration_s":0.0017139999999999998,"redis_read_bytes":9,"redis_write_bytes":1299,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00035,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000996,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000368,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012826,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026448,"duration_s":0.022636,"completed_at":"2026-11-13T10:37:09.292Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000342,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1255gitlab # [ 220.603763] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.293Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"5ce3a8c1fbbed407f3c5aad3","created_at":"2026-11-13T10:37:08.280Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"52c04cf6125c2a801631a7664dd30f97","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:08.294Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-5ce3a8c1fbbed407f3c5aad3: start","job_status":"start","queue_duration_s":0.998772,"scheduling_latency_s":0.998789}1256gitlab # [ 220.618596] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.306Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1257gitlab # [ 220.621936] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.315Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"5ce3a8c1fbbed407f3c5aad3","created_at":"2026-11-13T10:37:08.280Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"52c04cf6125c2a801631a7664dd30f97","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:08.294Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-5ce3a8c1fbbed407f3c5aad3: done: 0.022066 sec","job_status":"done","queue_duration_s":0.998772,"scheduling_latency_s":0.998789,"redis_calls":10,"redis_duration_s":0.0017729999999999998,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000352,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001029,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000392,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012569,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026688,"duration_s":0.022066,"completed_at":"2026-11-13T10:37:09.315Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000355,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1258gitlab # [ 220.666171] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.317Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"4958364db03c74678164ed55","created_at":"2026-11-13T10:37:08.306Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0bb5175663e785a44a31effd18f7103d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:08.316Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-4958364db03c74678164ed55: start","job_status":"start","queue_duration_s":1.000815,"scheduling_latency_s":1.000833}1259gitlab # [ 220.680715] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.580Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"4958364db03c74678164ed55","created_at":"2026-11-13T10:37:08.306Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0bb5175663e785a44a31effd18f7103d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:08.316Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-4958364db03c74678164ed55: done: 0.263264 sec","job_status":"done","queue_duration_s":1.000815,"scheduling_latency_s":1.000833,"redis_calls":12,"redis_duration_s":0.004209,"redis_read_bytes":11,"redis_write_bytes":1370,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000596,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001491,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002122,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.034,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.031,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.205916,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00127944,"exclusive_lock_hold_duration_s":0.23008989999999999,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.263264,"completed_at":"2026-11-13T10:37:09.580Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.053673,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1260gitlab # [ 220.727296] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.582Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"825941aad6b4590d8358cd35","created_at":"2026-11-13T10:37:08.332Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6899923d6e6950a32199727af1c595f7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:08.351Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-825941aad6b4590d8358cd35: start","job_status":"start","queue_duration_s":1.230793,"scheduling_latency_s":1.230813}1261gitlab # [ 220.742349] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.596Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1262gitlab # [ 220.746395] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.603Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"825941aad6b4590d8358cd35","created_at":"2026-11-13T10:37:08.332Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6899923d6e6950a32199727af1c595f7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:08.351Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-825941aad6b4590d8358cd35: done: 0.020461 sec","job_status":"done","queue_duration_s":1.230793,"scheduling_latency_s":1.230813,"redis_calls":10,"redis_duration_s":0.0020080000000000002,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000524,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00106,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000424,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012155,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00028172,"duration_s":0.020461,"completed_at":"2026-11-13T10:37:09.602Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000483,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1263gitlab # [ 220.791291] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.604Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"c2beb7b3be5fdfc318957b5e","created_at":"2026-11-13T10:37:08.387Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:08.401Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-c2beb7b3be5fdfc318957b5e: start","job_status":"start","queue_duration_s":1.202498,"scheduling_latency_s":1.202517}1264gitlab # [ 220.805465] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.724Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"c2beb7b3be5fdfc318957b5e","created_at":"2026-11-13T10:37:08.387Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:08.401Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-c2beb7b3be5fdfc318957b5e: done: 0.119711 sec","job_status":"done","queue_duration_s":1.202498,"scheduling_latency_s":1.202517,"redis_calls":71,"redis_duration_s":0.018278,"redis_read_bytes":120,"redis_write_bytes":42674,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.007631,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000726,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.009341,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00058,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.099371,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00027522,"duration_s":0.119711,"completed_at":"2026-11-13T10:37:09.724Z","load_balancing_strategy":"primary","db_duration_s":0.002817,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1265gitlab # [ 220.849293] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.726Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"eacb7cb591cf4149cb058782","created_at":"2026-11-13T10:37:08.425Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"79a2bf4fe9467b7c15cde44e66534b24","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:08.433Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-eacb7cb591cf4149cb058782: start","job_status":"start","queue_duration_s":1.292744,"scheduling_latency_s":1.29277}1266gitlab # [ 220.863785] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.752Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"eacb7cb591cf4149cb058782","created_at":"2026-11-13T10:37:08.425Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"79a2bf4fe9467b7c15cde44e66534b24","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:08.433Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-eacb7cb591cf4149cb058782: done: 0.026869 sec","job_status":"done","queue_duration_s":1.292744,"scheduling_latency_s":1.29277,"redis_calls":10,"redis_duration_s":0.0017369999999999998,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000334,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001055,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000348,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016369,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002586,"duration_s":0.026869,"completed_at":"2026-11-13T10:37:09.752Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003551,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1267gitlab # [ 220.907247] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.754Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"04cc003e6e2e48d19fcac76c","created_at":"2026-11-13T10:37:09.131Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d3a5489379c44b1d","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"11db46fe885cc708b1e6891f00123615","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.146Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-04cc003e6e2e48d19fcac76c: start","job_status":"start","queue_duration_s":0.608139,"scheduling_latency_s":0.60816}1268gitlab # [ 220.920004] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.771Z","message":"No existing merge request to be cleaned up."}1269gitlab # [ 220.921781] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.784Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"04cc003e6e2e48d19fcac76c","created_at":"2026-11-13T10:37:09.131Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d3a5489379c44b1d","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"11db46fe885cc708b1e6891f00123615","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.146Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-04cc003e6e2e48d19fcac76c: done: 0.030602 sec","job_status":"done","queue_duration_s":0.608139,"scheduling_latency_s":0.60816,"redis_calls":13,"redis_duration_s":0.002118,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000303,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000743,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001072,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02026,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00029032,"duration_s":0.030602,"completed_at":"2026-11-13T10:37:09.784Z","load_balancing_strategy":"primary","db_duration_s":0.004028,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1270gitlab # [ 220.963765] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.786Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"afe5854c676036ecc0d42e6d","created_at":"2026-11-13T10:37:09.131Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d3a5489379c44b1d","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"11db46fe885cc708b1e6891f00123615","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.146Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-afe5854c676036ecc0d42e6d: start","job_status":"start","queue_duration_s":0.640115,"scheduling_latency_s":0.640134}1271gitlab # [ 220.976665] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.797Z","message":"No existing merge request to be cleaned up."}1272gitlab # [ 220.978327] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.806Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"afe5854c676036ecc0d42e6d","created_at":"2026-11-13T10:37:09.131Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d3a5489379c44b1d","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"11db46fe885cc708b1e6891f00123615","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.146Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-afe5854c676036ecc0d42e6d: done: 0.020284 sec","job_status":"done","queue_duration_s":0.640115,"scheduling_latency_s":0.640134,"redis_calls":10,"redis_duration_s":0.0015170000000000001,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000629,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000888,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014224,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020284,"completed_at":"2026-11-13T10:37:09.806Z","load_balancing_strategy":"primary","db_duration_s":0.001761,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1273gitlab # [ 221.018207] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.807Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"8e7bd31e58648c9f4cf19ecc","created_at":"2026-11-13T10:37:09.131Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d3a5489379c44b1d","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"11db46fe885cc708b1e6891f00123615","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.146Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-8e7bd31e58648c9f4cf19ecc: start","job_status":"start","queue_duration_s":0.66167,"scheduling_latency_s":0.66169}1274gitlab # [ 221.030867] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.818Z","message":"No existing merge request to be cleaned up."}1275gitlab # [ 221.032582] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.827Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"8e7bd31e58648c9f4cf19ecc","created_at":"2026-11-13T10:37:09.131Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d3a5489379c44b1d","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"11db46fe885cc708b1e6891f00123615","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.146Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-8e7bd31e58648c9f4cf19ecc: done: 0.019878 sec","job_status":"done","queue_duration_s":0.66167,"scheduling_latency_s":0.66169,"redis_calls":10,"redis_duration_s":0.00149,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000652,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000838,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013921,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019878,"completed_at":"2026-11-13T10:37:09.827Z","load_balancing_strategy":"primary","db_duration_s":0.001498,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1276gitlab # [ 221.072783] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.829Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d901c9491d78832cdd2d09a6","created_at":"2026-11-13T10:37:09.131Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d3a5489379c44b1d","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"11db46fe885cc708b1e6891f00123615","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.146Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d901c9491d78832cdd2d09a6: start","job_status":"start","queue_duration_s":0.682795,"scheduling_latency_s":0.682813}1277gitlab # [ 221.085534] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.839Z","message":"No existing merge request to be cleaned up."}1278gitlab # [ 221.087230] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.847Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d901c9491d78832cdd2d09a6","created_at":"2026-11-13T10:37:09.131Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d3a5489379c44b1d","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"11db46fe885cc708b1e6891f00123615","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.146Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d901c9491d78832cdd2d09a6: done: 0.018632 sec","job_status":"done","queue_duration_s":0.682795,"scheduling_latency_s":0.682813,"redis_calls":10,"redis_duration_s":0.0014299999999999998,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000602,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000828,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013135,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018632,"completed_at":"2026-11-13T10:37:09.847Z","load_balancing_strategy":"primary","db_duration_s":0.001374,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1279gitlab # [ 221.127361] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.849Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d33d54c0a09bee14654aacb","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d33d54c0a09bee14654aacb: start","job_status":"start","queue_duration_s":0.134227,"scheduling_latency_s":0.134247}1280gitlab # [ 221.140279] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.871Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d33d54c0a09bee14654aacb","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d33d54c0a09bee14654aacb: done: 0.022772 sec","job_status":"done","queue_duration_s":0.134227,"scheduling_latency_s":0.134247,"redis_calls":16,"redis_duration_s":0.002084,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000293,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000619,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001172,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01438,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00024782,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022772,"completed_at":"2026-11-13T10:37:09.871Z","load_balancing_strategy":"primary","db_duration_s":0.000376,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1281gitlab # [ 221.182973] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.873Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be5e535ffd553d0680f6a195","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be5e535ffd553d0680f6a195: start","job_status":"start","queue_duration_s":0.158271,"scheduling_latency_s":0.158289}1282gitlab # [ 221.196080] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.888Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be5e535ffd553d0680f6a195","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be5e535ffd553d0680f6a195: done: 0.015661 sec","job_status":"done","queue_duration_s":0.158271,"scheduling_latency_s":0.158289,"redis_calls":13,"redis_duration_s":0.001618,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000581,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001037,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010627,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015661,"completed_at":"2026-11-13T10:37:09.888Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1283gitlab # [ 221.236615] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.890Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"addbf41dff5ec9779a057a68","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-addbf41dff5ec9779a057a68: start","job_status":"start","queue_duration_s":0.175146,"scheduling_latency_s":0.175165}1284gitlab # [ 221.249387] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.905Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"addbf41dff5ec9779a057a68","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-addbf41dff5ec9779a057a68: done: 0.015281 sec","job_status":"done","queue_duration_s":0.175146,"scheduling_latency_s":0.175165,"redis_calls":13,"redis_duration_s":0.0015609999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000565,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000996,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010553,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015281,"completed_at":"2026-11-13T10:37:09.905Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1285gitlab # [ 221.289690] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.906Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cbdef3c3964380da16b3b9ff","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cbdef3c3964380da16b3b9ff: start","job_status":"start","queue_duration_s":0.191789,"scheduling_latency_s":0.191809}1286gitlab # [ 221.302427] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.922Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cbdef3c3964380da16b3b9ff","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cbdef3c3964380da16b3b9ff: done: 0.015658 sec","job_status":"done","queue_duration_s":0.191789,"scheduling_latency_s":0.191809,"redis_calls":13,"redis_duration_s":0.0016220000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000571,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001051,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010877,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015658,"completed_at":"2026-11-13T10:37:09.922Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1287gitlab # [ 221.343112] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.923Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5fc3846cea3a50269de939fa","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5fc3846cea3a50269de939fa: start","job_status":"start","queue_duration_s":0.208794,"scheduling_latency_s":0.208812}1288gitlab # [ 221.355809] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.939Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5fc3846cea3a50269de939fa","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5fc3846cea3a50269de939fa: done: 0.015797 sec","job_status":"done","queue_duration_s":0.208794,"scheduling_latency_s":0.208812,"redis_calls":13,"redis_duration_s":0.0015769999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00056,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001017,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011059,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015797,"completed_at":"2026-11-13T10:37:09.939Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1289gitlab # [ 221.397257] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.940Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"92f6dc62c2f7e08f6e9715d8","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-92f6dc62c2f7e08f6e9715d8: start","job_status":"start","queue_duration_s":0.225981,"scheduling_latency_s":0.226001}1290gitlab # [ 221.410078] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.956Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"92f6dc62c2f7e08f6e9715d8","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-92f6dc62c2f7e08f6e9715d8: done: 0.015411 sec","job_status":"done","queue_duration_s":0.225981,"scheduling_latency_s":0.226001,"redis_calls":13,"redis_duration_s":0.001549,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000553,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000996,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01092,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015411,"completed_at":"2026-11-13T10:37:09.956Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1291gitlab # [ 221.450392] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.957Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aebc05bee93d874249436f82","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aebc05bee93d874249436f82: start","job_status":"start","queue_duration_s":0.242845,"scheduling_latency_s":0.242866}1292gitlab # [ 221.463433] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.973Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aebc05bee93d874249436f82","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aebc05bee93d874249436f82: done: 0.015399 sec","job_status":"done","queue_duration_s":0.242845,"scheduling_latency_s":0.242866,"redis_calls":13,"redis_duration_s":0.001595,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000566,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001029,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010853,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015399,"completed_at":"2026-11-13T10:37:09.973Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1293gitlab # [ 221.504248] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.974Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8afa9307e1b6d136a256b1e6","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8afa9307e1b6d136a256b1e6: start","job_status":"start","queue_duration_s":0.259632,"scheduling_latency_s":0.259651}1294gitlab # [ 221.517143] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.989Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8afa9307e1b6d136a256b1e6","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8afa9307e1b6d136a256b1e6: done: 0.015176 sec","job_status":"done","queue_duration_s":0.259632,"scheduling_latency_s":0.259651,"redis_calls":13,"redis_duration_s":0.001539,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000547,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000992,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010688,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015176,"completed_at":"2026-11-13T10:37:09.989Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1295gitlab # [ 221.557949] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:09.990Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2e268b15616080810ba3e11b","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2e268b15616080810ba3e11b: start","job_status":"start","queue_duration_s":0.275967,"scheduling_latency_s":0.275986}1296gitlab # [ 221.571791] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.006Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2e268b15616080810ba3e11b","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2e268b15616080810ba3e11b: done: 0.015387 sec","job_status":"done","queue_duration_s":0.275967,"scheduling_latency_s":0.275986,"redis_calls":13,"redis_duration_s":0.0016259999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000569,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001057,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010718,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015387,"completed_at":"2026-11-13T10:37:10.006Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1297gitlab # [ 221.612669] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.007Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"19c240142cf0d1cc0009c3fa","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-19c240142cf0d1cc0009c3fa: start","job_status":"start","queue_duration_s":0.29255,"scheduling_latency_s":0.292569}1298gitlab # [ 221.625544] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.022Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"19c240142cf0d1cc0009c3fa","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-19c240142cf0d1cc0009c3fa: done: 0.015415 sec","job_status":"done","queue_duration_s":0.29255,"scheduling_latency_s":0.292569,"redis_calls":13,"redis_duration_s":0.0015970000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000577,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00102,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01062,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015415,"completed_at":"2026-11-13T10:37:10.022Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1299gitlab # [ 221.665959] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.024Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f5ff80608fbb7c2bb8a0379f","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f5ff80608fbb7c2bb8a0379f: start","job_status":"start","queue_duration_s":0.309133,"scheduling_latency_s":0.309151}1300gitlab # [ 221.678869] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.040Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f5ff80608fbb7c2bb8a0379f","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f5ff80608fbb7c2bb8a0379f: done: 0.016179 sec","job_status":"done","queue_duration_s":0.309133,"scheduling_latency_s":0.309151,"redis_calls":13,"redis_duration_s":0.00178,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000583,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001197,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010976,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016179,"completed_at":"2026-11-13T10:37:10.040Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1301gitlab # [ 221.719972] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.041Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e959fa64944b3173bd999889","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e959fa64944b3173bd999889: start","job_status":"start","queue_duration_s":0.326482,"scheduling_latency_s":0.3265}1302gitlab # [ 221.732855] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.056Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e959fa64944b3173bd999889","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e959fa64944b3173bd999889: done: 0.015532 sec","job_status":"done","queue_duration_s":0.326482,"scheduling_latency_s":0.3265,"redis_calls":13,"redis_duration_s":0.001595,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000575,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00102,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01071,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015532,"completed_at":"2026-11-13T10:37:10.056Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1303gitlab # [ 221.774442] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.058Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ffdee46829614ec7de08491e","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ffdee46829614ec7de08491e: start","job_status":"start","queue_duration_s":0.343248,"scheduling_latency_s":0.343266}1304gitlab # [ 221.787474] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.073Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ffdee46829614ec7de08491e","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ffdee46829614ec7de08491e: done: 0.015302 sec","job_status":"done","queue_duration_s":0.343248,"scheduling_latency_s":0.343266,"redis_calls":13,"redis_duration_s":0.001558,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000573,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000985,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010576,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015302,"completed_at":"2026-11-13T10:37:10.073Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1305gitlab # [ 221.828317] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.074Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7bf7bed82b8ef6d31983c7ff","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7bf7bed82b8ef6d31983c7ff: start","job_status":"start","queue_duration_s":0.359897,"scheduling_latency_s":0.359916}1306gitlab # [ 221.840847] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.090Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7bf7bed82b8ef6d31983c7ff","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7bf7bed82b8ef6d31983c7ff: done: 0.015789 sec","job_status":"done","queue_duration_s":0.359897,"scheduling_latency_s":0.359916,"redis_calls":13,"redis_duration_s":0.001605,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00059,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001015,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011101,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015789,"completed_at":"2026-11-13T10:37:10.090Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1307gitlab # [ 221.881164] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.091Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b763c40cd94c4bdfdcf29149","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b763c40cd94c4bdfdcf29149: start","job_status":"start","queue_duration_s":0.377002,"scheduling_latency_s":0.377021}1308gitlab # [ 221.894004] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.107Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b763c40cd94c4bdfdcf29149","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b763c40cd94c4bdfdcf29149: done: 0.015521 sec","job_status":"done","queue_duration_s":0.377002,"scheduling_latency_s":0.377021,"redis_calls":13,"redis_duration_s":0.001664,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000575,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001089,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010946,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015521,"completed_at":"2026-11-13T10:37:10.107Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1309gitlab # [ 221.934261] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.108Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ce7de12dc5330f94b3e656a4","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ce7de12dc5330f94b3e656a4: start","job_status":"start","queue_duration_s":0.393936,"scheduling_latency_s":0.393956}1310gitlab # [ 221.947027] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.124Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ce7de12dc5330f94b3e656a4","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ce7de12dc5330f94b3e656a4: done: 0.015575 sec","job_status":"done","queue_duration_s":0.393936,"scheduling_latency_s":0.393956,"redis_calls":13,"redis_duration_s":0.0015600000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000566,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000994,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011033,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015575,"completed_at":"2026-11-13T10:37:10.124Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1311gitlab # [ 221.988742] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.125Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"daea5872874b40451341689b","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-daea5872874b40451341689b: start","job_status":"start","queue_duration_s":0.410883,"scheduling_latency_s":0.410902}1312gitlab # [ 222.001502] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.141Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"daea5872874b40451341689b","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-daea5872874b40451341689b: done: 0.01561 sec","job_status":"done","queue_duration_s":0.410883,"scheduling_latency_s":0.410902,"redis_calls":13,"redis_duration_s":0.0015669999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000579,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000988,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010845,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01561,"completed_at":"2026-11-13T10:37:10.141Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1313gitlab # [ 222.042322] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.142Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cb30da6fd5da2b4078be98a7","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cb30da6fd5da2b4078be98a7: start","job_status":"start","queue_duration_s":0.427955,"scheduling_latency_s":0.427973}1314gitlab # [ 222.055259] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.158Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cb30da6fd5da2b4078be98a7","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cb30da6fd5da2b4078be98a7: done: 0.015544 sec","job_status":"done","queue_duration_s":0.427955,"scheduling_latency_s":0.427973,"redis_calls":13,"redis_duration_s":0.001799,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000774,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001025,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010961,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015544,"completed_at":"2026-11-13T10:37:10.158Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1315gitlab # [ 222.096184] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.159Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1e5ad8821e91e47fac9d3f13","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1e5ad8821e91e47fac9d3f13: start","job_status":"start","queue_duration_s":0.444721,"scheduling_latency_s":0.444742}1316gitlab # [ 222.109048] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.175Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1e5ad8821e91e47fac9d3f13","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1e5ad8821e91e47fac9d3f13: done: 0.015825 sec","job_status":"done","queue_duration_s":0.444721,"scheduling_latency_s":0.444742,"redis_calls":13,"redis_duration_s":0.0016640000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000585,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001079,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011025,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015825,"completed_at":"2026-11-13T10:37:10.175Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1317gitlab # [ 222.149264] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.176Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"79791222015f258ef1cf1cfd","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-79791222015f258ef1cf1cfd: start","job_status":"start","queue_duration_s":0.461728,"scheduling_latency_s":0.461747}1318gitlab # [ 222.161982] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.191Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"79791222015f258ef1cf1cfd","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-79791222015f258ef1cf1cfd: done: 0.015307 sec","job_status":"done","queue_duration_s":0.461728,"scheduling_latency_s":0.461747,"redis_calls":13,"redis_duration_s":0.0017399999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000571,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001169,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010473,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015307,"completed_at":"2026-11-13T10:37:10.191Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1319gitlab # [ 222.203904] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.193Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"97acc94a9604a9879a4c24ae","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-97acc94a9604a9879a4c24ae: start","job_status":"start","queue_duration_s":0.478244,"scheduling_latency_s":0.478263}1320gitlab # [ 222.216969] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.208Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"97acc94a9604a9879a4c24ae","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-97acc94a9604a9879a4c24ae: done: 0.015575 sec","job_status":"done","queue_duration_s":0.478244,"scheduling_latency_s":0.478263,"redis_calls":13,"redis_duration_s":0.001661,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000554,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001107,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010645,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015575,"completed_at":"2026-11-13T10:37:10.208Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1321gitlab # [ 222.257038] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.209Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7ecfd4077a0fcafd49de42da","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7ecfd4077a0fcafd49de42da: start","job_status":"start","queue_duration_s":0.49502,"scheduling_latency_s":0.495038}1322gitlab # [ 222.270049] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.225Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7ecfd4077a0fcafd49de42da","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7ecfd4077a0fcafd49de42da: done: 0.015464 sec","job_status":"done","queue_duration_s":0.49502,"scheduling_latency_s":0.495038,"redis_calls":13,"redis_duration_s":0.001549,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000563,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000986,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010618,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015464,"completed_at":"2026-11-13T10:37:10.225Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1323gitlab # [ 222.310287] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.226Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"595a7d3e0355453e1acfc215","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-595a7d3e0355453e1acfc215: start","job_status":"start","queue_duration_s":0.511708,"scheduling_latency_s":0.511727}1324gitlab # [ 222.323092] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.242Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"595a7d3e0355453e1acfc215","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-595a7d3e0355453e1acfc215: done: 0.016024 sec","job_status":"done","queue_duration_s":0.511708,"scheduling_latency_s":0.511727,"redis_calls":13,"redis_duration_s":0.001714,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000616,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001098,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010861,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016024,"completed_at":"2026-11-13T10:37:10.242Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1325gitlab # [ 222.363541] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.243Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5183b198709ee0a6930af7de","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5183b198709ee0a6930af7de: start","job_status":"start","queue_duration_s":0.529023,"scheduling_latency_s":0.529042}1326gitlab # [ 222.376515] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.259Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5183b198709ee0a6930af7de","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5183b198709ee0a6930af7de: done: 0.015213 sec","job_status":"done","queue_duration_s":0.529023,"scheduling_latency_s":0.529042,"redis_calls":13,"redis_duration_s":0.001533,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000557,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000976,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010571,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015213,"completed_at":"2026-11-13T10:37:10.259Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1327gitlab # [ 222.417486] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.260Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"52102db16559a2ed29175ea1","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-52102db16559a2ed29175ea1: start","job_status":"start","queue_duration_s":0.545609,"scheduling_latency_s":0.545627}1328gitlab # [ 222.430076] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.275Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"52102db16559a2ed29175ea1","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-52102db16559a2ed29175ea1: done: 0.015228 sec","job_status":"done","queue_duration_s":0.545609,"scheduling_latency_s":0.545627,"redis_calls":13,"redis_duration_s":0.001594,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000604,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00099,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010785,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015228,"completed_at":"2026-11-13T10:37:10.275Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1329gitlab # [ 222.470273] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.277Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b74ef32999bc6ddb2da88ab1","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b74ef32999bc6ddb2da88ab1: start","job_status":"start","queue_duration_s":0.562196,"scheduling_latency_s":0.562217}1330gitlab # [ 222.483177] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.292Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b74ef32999bc6ddb2da88ab1","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b74ef32999bc6ddb2da88ab1: done: 0.015579 sec","job_status":"done","queue_duration_s":0.562196,"scheduling_latency_s":0.562217,"redis_calls":13,"redis_duration_s":0.001564,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000562,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001002,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011003,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015579,"completed_at":"2026-11-13T10:37:10.292Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1331gitlab # [ 222.523490] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.294Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ace05f407e3bb298ce72fcc4","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ace05f407e3bb298ce72fcc4: start","job_status":"start","queue_duration_s":0.579162,"scheduling_latency_s":0.579181}1332gitlab # [ 222.536383] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.313Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ace05f407e3bb298ce72fcc4","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ace05f407e3bb298ce72fcc4: done: 0.019014 sec","job_status":"done","queue_duration_s":0.579162,"scheduling_latency_s":0.579181,"redis_calls":13,"redis_duration_s":0.001564,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000595,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000969,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010952,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019014,"completed_at":"2026-11-13T10:37:10.312Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1333gitlab # [ 222.576659] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.314Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d22287c87460c2d33a9be5db","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d22287c87460c2d33a9be5db: start","job_status":"start","queue_duration_s":0.599694,"scheduling_latency_s":0.599714}1334gitlab # [ 222.590563] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.329Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d22287c87460c2d33a9be5db","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d22287c87460c2d33a9be5db: done: 0.01547 sec","job_status":"done","queue_duration_s":0.599694,"scheduling_latency_s":0.599714,"redis_calls":13,"redis_duration_s":0.001549,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000562,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000987,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010968,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01547,"completed_at":"2026-11-13T10:37:10.329Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1335gitlab # [ 222.630972] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.331Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1a69739ebbe6ee1da4db891a","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1a69739ebbe6ee1da4db891a: start","job_status":"start","queue_duration_s":0.616678,"scheduling_latency_s":0.616698}1336gitlab # [ 222.643708] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.348Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1a69739ebbe6ee1da4db891a","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1a69739ebbe6ee1da4db891a: done: 0.016886 sec","job_status":"done","queue_duration_s":0.616678,"scheduling_latency_s":0.616698,"redis_calls":13,"redis_duration_s":0.0017410000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000644,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001097,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011791,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016886,"completed_at":"2026-11-13T10:37:10.348Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1337gitlab # [ 222.684372] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.349Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a2556a25dce6465df8e7f734","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a2556a25dce6465df8e7f734: start","job_status":"start","queue_duration_s":0.634845,"scheduling_latency_s":0.634864}1338gitlab # [ 222.697253] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.365Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a2556a25dce6465df8e7f734","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a2556a25dce6465df8e7f734: done: 0.015978 sec","job_status":"done","queue_duration_s":0.634845,"scheduling_latency_s":0.634864,"redis_calls":13,"redis_duration_s":0.001655,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000554,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001101,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011116,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015978,"completed_at":"2026-11-13T10:37:10.365Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1339gitlab # [ 222.737251] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.366Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b1fcf7ea299d6da2d665f0e1","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b1fcf7ea299d6da2d665f0e1: start","job_status":"start","queue_duration_s":0.652011,"scheduling_latency_s":0.65203}1340gitlab # [ 222.750024] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.382Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b1fcf7ea299d6da2d665f0e1","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b1fcf7ea299d6da2d665f0e1: done: 0.015804 sec","job_status":"done","queue_duration_s":0.652011,"scheduling_latency_s":0.65203,"redis_calls":13,"redis_duration_s":0.001617,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000575,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001042,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010838,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015804,"completed_at":"2026-11-13T10:37:10.382Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1341gitlab # [ 222.791609] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.383Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c0fc41b2bf2f56198b02207c","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c0fc41b2bf2f56198b02207c: start","job_status":"start","queue_duration_s":0.669064,"scheduling_latency_s":0.669083}1342gitlab # [ 222.804665] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.399Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c0fc41b2bf2f56198b02207c","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c0fc41b2bf2f56198b02207c: done: 0.015953 sec","job_status":"done","queue_duration_s":0.669064,"scheduling_latency_s":0.669083,"redis_calls":13,"redis_duration_s":0.0016380000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000572,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001066,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010948,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015953,"completed_at":"2026-11-13T10:37:10.399Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1343gitlab # [ 222.845302] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.401Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e59e69694296184e2c70aa9","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e59e69694296184e2c70aa9: start","job_status":"start","queue_duration_s":0.686215,"scheduling_latency_s":0.686235}1344gitlab # [ 222.857993] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.416Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e59e69694296184e2c70aa9","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e59e69694296184e2c70aa9: done: 0.015661 sec","job_status":"done","queue_duration_s":0.686215,"scheduling_latency_s":0.686235,"redis_calls":13,"redis_duration_s":0.0016120000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000571,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001041,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010681,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015661,"completed_at":"2026-11-13T10:37:10.416Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1345gitlab # [ 222.898480] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.417Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"21dbd311d5b7b0145b0261b7","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-21dbd311d5b7b0145b0261b7: start","job_status":"start","queue_duration_s":0.703094,"scheduling_latency_s":0.703113}1346gitlab # [ 222.911428] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.434Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"21dbd311d5b7b0145b0261b7","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-21dbd311d5b7b0145b0261b7: done: 0.016217 sec","job_status":"done","queue_duration_s":0.703094,"scheduling_latency_s":0.703113,"redis_calls":13,"redis_duration_s":0.0019690000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000908,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001061,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010972,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016217,"completed_at":"2026-11-13T10:37:10.434Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1347gitlab # [ 222.952105] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.435Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e5bc99f8f49146a11b855db7","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e5bc99f8f49146a11b855db7: start","job_status":"start","queue_duration_s":0.720742,"scheduling_latency_s":0.720762}1348gitlab # [ 222.964697] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.451Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e5bc99f8f49146a11b855db7","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e5bc99f8f49146a11b855db7: done: 0.015911 sec","job_status":"done","queue_duration_s":0.720742,"scheduling_latency_s":0.720762,"redis_calls":13,"redis_duration_s":0.001703,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000586,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001117,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01099,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015911,"completed_at":"2026-11-13T10:37:10.451Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1349gitlab # [ 223.005879] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.452Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e478c581ad4b3ad8711aaf7","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e478c581ad4b3ad8711aaf7: start","job_status":"start","queue_duration_s":0.738052,"scheduling_latency_s":0.738071}1350gitlab # [ 223.018556] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.468Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e478c581ad4b3ad8711aaf7","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e478c581ad4b3ad8711aaf7: done: 0.015858 sec","job_status":"done","queue_duration_s":0.738052,"scheduling_latency_s":0.738071,"redis_calls":13,"redis_duration_s":0.0018189999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000772,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001047,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011183,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015858,"completed_at":"2026-11-13T10:37:10.468Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1351gitlab # [ 223.059110] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.470Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34dd936c3def6daa44eeaf3c","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34dd936c3def6daa44eeaf3c: start","job_status":"start","queue_duration_s":0.755373,"scheduling_latency_s":0.755394}1352gitlab # [ 223.071892] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.486Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34dd936c3def6daa44eeaf3c","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34dd936c3def6daa44eeaf3c: done: 0.016778 sec","job_status":"done","queue_duration_s":0.755373,"scheduling_latency_s":0.755394,"redis_calls":13,"redis_duration_s":0.001768,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000657,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001111,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011875,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016778,"completed_at":"2026-11-13T10:37:10.486Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1353gitlab # [ 223.112982] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.488Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"56e4ccc7169b9b12c0819b6e","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-56e4ccc7169b9b12c0819b6e: start","job_status":"start","queue_duration_s":0.773644,"scheduling_latency_s":0.773664}1354gitlab # [ 223.125709] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.503Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"56e4ccc7169b9b12c0819b6e","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-56e4ccc7169b9b12c0819b6e: done: 0.015259 sec","job_status":"done","queue_duration_s":0.773644,"scheduling_latency_s":0.773664,"redis_calls":13,"redis_duration_s":0.001563,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00057,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000993,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0108,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015259,"completed_at":"2026-11-13T10:37:10.503Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1355gitlab # [ 223.165699] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.504Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ec0164c99998c948c3b4743b","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ec0164c99998c948c3b4743b: start","job_status":"start","queue_duration_s":0.790096,"scheduling_latency_s":0.790117}1356gitlab # [ 223.178429] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.520Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ec0164c99998c948c3b4743b","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ec0164c99998c948c3b4743b: done: 0.015533 sec","job_status":"done","queue_duration_s":0.790096,"scheduling_latency_s":0.790117,"redis_calls":13,"redis_duration_s":0.0015890000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000548,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001041,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010739,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015533,"completed_at":"2026-11-13T10:37:10.520Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1357gitlab # [ 223.219991] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.521Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"236b6a15d8b7b6c041a5e2f8","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-236b6a15d8b7b6c041a5e2f8: start","job_status":"start","queue_duration_s":0.806883,"scheduling_latency_s":0.806902}1358gitlab # [ 223.232550] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.537Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"236b6a15d8b7b6c041a5e2f8","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-236b6a15d8b7b6c041a5e2f8: done: 0.01588 sec","job_status":"done","queue_duration_s":0.806883,"scheduling_latency_s":0.806902,"redis_calls":13,"redis_duration_s":0.00183,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000552,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001278,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010883,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01588,"completed_at":"2026-11-13T10:37:10.537Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1359gitlab # [ 223.272761] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.538Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4a18c82d6aa3908d3a2a9d72","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4a18c82d6aa3908d3a2a9d72: start","job_status":"start","queue_duration_s":0.823929,"scheduling_latency_s":0.823948}1360gitlab # [ 223.285285] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.554Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4a18c82d6aa3908d3a2a9d72","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4a18c82d6aa3908d3a2a9d72: done: 0.015468 sec","job_status":"done","queue_duration_s":0.823929,"scheduling_latency_s":0.823948,"redis_calls":13,"redis_duration_s":0.001557,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000559,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000998,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010599,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015468,"completed_at":"2026-11-13T10:37:10.554Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1361gitlab # [ 223.325337] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.555Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"40aec758b6778b527dd148f8","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-40aec758b6778b527dd148f8: start","job_status":"start","queue_duration_s":0.840586,"scheduling_latency_s":0.840604}1362gitlab # [ 223.338191] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.570Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"40aec758b6778b527dd148f8","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-40aec758b6778b527dd148f8: done: 0.015443 sec","job_status":"done","queue_duration_s":0.840586,"scheduling_latency_s":0.840604,"redis_calls":13,"redis_duration_s":0.001565,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000583,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000982,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010597,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015443,"completed_at":"2026-11-13T10:37:10.570Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1363gitlab # [ 223.378398] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.572Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12ced26185dc920977177b13","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12ced26185dc920977177b13: start","job_status":"start","queue_duration_s":0.857224,"scheduling_latency_s":0.857245}1364gitlab # [ 223.391201] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.587Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12ced26185dc920977177b13","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12ced26185dc920977177b13: done: 0.015485 sec","job_status":"done","queue_duration_s":0.857224,"scheduling_latency_s":0.857245,"redis_calls":13,"redis_duration_s":0.001584,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000561,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001023,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010612,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015485,"completed_at":"2026-11-13T10:37:10.587Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1365gitlab # [ 223.432528] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.588Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3b3c85b62d458d0af9fad98","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3b3c85b62d458d0af9fad98: start","job_status":"start","queue_duration_s":0.873936,"scheduling_latency_s":0.873956}1366gitlab # [ 223.445082] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.604Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3b3c85b62d458d0af9fad98","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3b3c85b62d458d0af9fad98: done: 0.015366 sec","job_status":"done","queue_duration_s":0.873936,"scheduling_latency_s":0.873956,"redis_calls":13,"redis_duration_s":0.001547,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000554,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000993,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01067,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015366,"completed_at":"2026-11-13T10:37:10.604Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1367gitlab # [ 223.485156] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.605Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"23fcb1221d6e4be24298af81","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-23fcb1221d6e4be24298af81: start","job_status":"start","queue_duration_s":0.890656,"scheduling_latency_s":0.890676}1368gitlab # [ 223.497900] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.620Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"23fcb1221d6e4be24298af81","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-23fcb1221d6e4be24298af81: done: 0.015241 sec","job_status":"done","queue_duration_s":0.890656,"scheduling_latency_s":0.890676,"redis_calls":13,"redis_duration_s":0.001529,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000544,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000985,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010631,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015241,"completed_at":"2026-11-13T10:37:10.620Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1369gitlab # [ 223.538114] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.622Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b16e1ab8e9763c35c2eba7b6","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b16e1ab8e9763c35c2eba7b6: start","job_status":"start","queue_duration_s":0.9072,"scheduling_latency_s":0.907219}1370gitlab # [ 223.550748] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.637Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b16e1ab8e9763c35c2eba7b6","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b16e1ab8e9763c35c2eba7b6: done: 0.015705 sec","job_status":"done","queue_duration_s":0.9072,"scheduling_latency_s":0.907219,"redis_calls":13,"redis_duration_s":0.001578,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000571,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001007,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010948,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015705,"completed_at":"2026-11-13T10:37:10.637Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1371gitlab # [ 223.591105] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.639Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"23b7c50a0525c194f57c5f6b","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-23b7c50a0525c194f57c5f6b: start","job_status":"start","queue_duration_s":0.924272,"scheduling_latency_s":0.924292}1372gitlab # [ 223.604045] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.654Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"23b7c50a0525c194f57c5f6b","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-23b7c50a0525c194f57c5f6b: done: 0.015282 sec","job_status":"done","queue_duration_s":0.924272,"scheduling_latency_s":0.924292,"redis_calls":13,"redis_duration_s":0.001568,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000551,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001017,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010757,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015282,"completed_at":"2026-11-13T10:37:10.654Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1373gitlab # [ 223.644956] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.655Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"867cf90f37925a8575fdef81","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-867cf90f37925a8575fdef81: start","job_status":"start","queue_duration_s":0.940945,"scheduling_latency_s":0.940965}1374gitlab # [ 223.657517] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.671Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"867cf90f37925a8575fdef81","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-867cf90f37925a8575fdef81: done: 0.015893 sec","job_status":"done","queue_duration_s":0.940945,"scheduling_latency_s":0.940965,"redis_calls":13,"redis_duration_s":0.0016940000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000604,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00109,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011203,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015893,"completed_at":"2026-11-13T10:37:10.671Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1375gitlab # [ 223.697711] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.673Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d14712f5a4c6cd0cbbe67f9a","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d14712f5a4c6cd0cbbe67f9a: start","job_status":"start","queue_duration_s":0.958312,"scheduling_latency_s":0.958333}1376gitlab # [ 223.710467] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.689Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d14712f5a4c6cd0cbbe67f9a","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d14712f5a4c6cd0cbbe67f9a: done: 0.016546 sec","job_status":"done","queue_duration_s":0.958312,"scheduling_latency_s":0.958333,"redis_calls":13,"redis_duration_s":0.001824,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000573,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001251,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01179,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016546,"completed_at":"2026-11-13T10:37:10.689Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1377gitlab # [ 223.750760] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.690Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cbefdbb7f15affb9bb3c5111","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cbefdbb7f15affb9bb3c5111: start","job_status":"start","queue_duration_s":0.975999,"scheduling_latency_s":0.976018}1378gitlab # [ 223.763448] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:37:10.707Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cbefdbb7f15affb9bb3c5111","created_at":"2026-11-13T10:37:09.632Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-ef3c65f332dd436b","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c05493f91e5fc2e85a97eed6caf4d976","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:09.714Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cbefdbb7f15affb9bb3c5111: done: 0.017034 sec","job_status":"done","queue_duration_s":0.975999,"scheduling_latency_s":0.976018,"redis_calls":13,"redis_duration_s":0.0028190000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00142,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001399,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011483,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017034,"completed_at":"2026-11-13T10:37:10.707Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1379gitlab # [ 252.825589] setup-gitlab-start[2851]: W, [2026-11-13T10:37:41.774217 #2851] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1380gitlab # [ 275.201495] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:04.149Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"57bc8afe662feb9323b6b70a","created_at":"2026-11-13T10:38:04.130Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"05634b527103b827ed87bf039f0173f9","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:04.147Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-57bc8afe662feb9323b6b70a: start","job_status":"start","queue_duration_s":0.002049,"scheduling_latency_s":0.002085}1381gitlab # [ 275.354478] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:04.303Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"57bc8afe662feb9323b6b70a","created_at":"2026-11-13T10:38:04.130Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"05634b527103b827ed87bf039f0173f9","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:04.147Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-57bc8afe662feb9323b6b70a: done: 0.154064 sec","job_status":"done","queue_duration_s":0.002049,"scheduling_latency_s":0.002085,"redis_calls":17,"redis_duration_s":0.037752,"redis_read_bytes":904,"redis_write_bytes":1688,"redis_feature_flag_calls":7,"redis_feature_flag_duration_s":0.008479,"redis_feature_flag_read_bytes":896,"redis_feature_flag_write_bytes":449,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.022785,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.006488,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029525,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0179438,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.154064,"completed_at":"2026-11-13T10:38:04.302Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011981,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1382gitlab # [ 275.418832] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:04.346Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"bf24537ba5d28c9b05807ca1","created_at":"2026-11-13T10:38:04.193Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4f7852b65cce49f274781cf967f4bfab","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:04.198Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-bf24537ba5d28c9b05807ca1: start","job_status":"start","queue_duration_s":0.147476,"scheduling_latency_s":0.147517}1383gitlab # [ 275.621667] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:04.570Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"bf24537ba5d28c9b05807ca1","created_at":"2026-11-13T10:38:04.193Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4f7852b65cce49f274781cf967f4bfab","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:04.198Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-bf24537ba5d28c9b05807ca1: done: 0.224112 sec","job_status":"done","queue_duration_s":0.147476,"scheduling_latency_s":0.147517,"redis_calls":10,"redis_duration_s":0.012877,"redis_read_bytes":11,"redis_write_bytes":1166,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.007387,"redis_queues_metadata_read_bytes":8,"redis_queues_metadata_write_bytes":555,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00549,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.041,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.041,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037008,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00175526,"exclusive_lock_hold_duration_s":0.08263622,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.224112,"completed_at":"2026-11-13T10:38:04.570Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.06987,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1384gitlab # [ 275.690301] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:04.575Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"b5095057d57f9d40f67adc97","created_at":"2026-11-13T10:38:04.205Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8ce6779b545a9769b5e9558392f6278b","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:04.209Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-b5095057d57f9d40f67adc97: start","job_status":"start","queue_duration_s":0.365363,"scheduling_latency_s":0.365392}1385gitlab # [ 275.707893] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:04.637Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"b5095057d57f9d40f67adc97","created_at":"2026-11-13T10:38:04.205Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8ce6779b545a9769b5e9558392f6278b","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:04.209Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-b5095057d57f9d40f67adc97: done: 0.062651 sec","job_status":"done","queue_duration_s":0.365363,"scheduling_latency_s":0.365392,"redis_calls":9,"redis_duration_s":0.011214,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.007764,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00345,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017412,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00139504,"exclusive_lock_hold_duration_s":0.01292662,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.062651,"completed_at":"2026-11-13T10:38:04.637Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010696,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1386gitlab # [ 275.751598] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:04.643Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"9ea448a38ea1879e6082789f","created_at":"2026-11-13T10:38:04.219Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"beb59e78b0623a2a10153dfc38fd30b1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:04.229Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-9ea448a38ea1879e6082789f: start","job_status":"start","queue_duration_s":0.414355,"scheduling_latency_s":0.414383}1387gitlab # [ 275.766668] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:04.702Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"9ea448a38ea1879e6082789f","created_at":"2026-11-13T10:38:04.219Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"beb59e78b0623a2a10153dfc38fd30b1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:04.229Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-9ea448a38ea1879e6082789f: done: 0.059281 sec","job_status":"done","queue_duration_s":0.414355,"scheduling_latency_s":0.414383,"redis_calls":8,"redis_duration_s":0.013679,"redis_read_bytes":133,"redis_write_bytes":846,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000942,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":70,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.008274,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004463,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014701,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.059281,"completed_at":"2026-11-13T10:38:04.702Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1388gitlab # [ 275.810433] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:04.710Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"83e5be6195add7792b3a7d96","created_at":"2026-11-13T10:38:04.246Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c20b13b28d9095f4df62aa9d70030323","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:04.254Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-83e5be6195add7792b3a7d96: start","job_status":"start","queue_duration_s":0.455982,"scheduling_latency_s":0.456008}1389gitlab # [ 275.829507] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:04.778Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"83e5be6195add7792b3a7d96","created_at":"2026-11-13T10:38:04.246Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c20b13b28d9095f4df62aa9d70030323","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:04.254Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-83e5be6195add7792b3a7d96: done: 0.0682 sec","job_status":"done","queue_duration_s":0.455982,"scheduling_latency_s":0.456008,"redis_calls":7,"redis_duration_s":0.0059819999999999995,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004691,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001291,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017671,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0682,"completed_at":"2026-11-13T10:38:04.778Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.030038,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1390gitlab # [ 275.872121] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:04.780Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"8bfec931e73a0dbe2c70a8c4","created_at":"2026-11-13T10:38:04.282Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f319e1c4aae8ddbcaf3c1db2afcbd9d2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:04.288Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-8bfec931e73a0dbe2c70a8c4: start","job_status":"start","queue_duration_s":0.491686,"scheduling_latency_s":0.491711}1391gitlab # [ 275.886071] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:04.802Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"8bfec931e73a0dbe2c70a8c4","created_at":"2026-11-13T10:38:04.282Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f319e1c4aae8ddbcaf3c1db2afcbd9d2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:04.288Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-8bfec931e73a0dbe2c70a8c4: done: 0.022548 sec","job_status":"done","queue_duration_s":0.491686,"scheduling_latency_s":0.491711,"redis_calls":9,"redis_duration_s":0.00199,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001061,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000929,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014986,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0006816,"exclusive_lock_hold_duration_s":0.00434414,"duration_s":0.022548,"completed_at":"2026-11-13T10:38:04.802Z","load_balancing_strategy":"primary","db_duration_s":0.001464,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1392gitlab # [ 275.927001] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:04.804Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"3f96d65660ae1c6137aae2fa","created_at":"2026-11-13T10:38:04.298Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bd86e38b280942737e54bdf3d873dbbe","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:04.301Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-3f96d65660ae1c6137aae2fa: start","job_status":"start","queue_duration_s":0.503188,"scheduling_latency_s":0.503211}1393gitlab # [ 275.941496] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:04.830Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"3f96d65660ae1c6137aae2fa","created_at":"2026-11-13T10:38:04.298Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bd86e38b280942737e54bdf3d873dbbe","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:04.301Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-3f96d65660ae1c6137aae2fa: done: 0.025912 sec","job_status":"done","queue_duration_s":0.503188,"scheduling_latency_s":0.503211,"redis_calls":7,"redis_duration_s":0.0015069999999999999,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001099,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000408,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012166,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.025912,"completed_at":"2026-11-13T10:38:04.830Z","load_balancing_strategy":"primary","db_duration_s":0.008047,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1394gitlab # [ 275.983535] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:04.831Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"bc330c13b9c25f1f8495be73","created_at":"2026-11-13T10:38:04.347Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d183e4ce2f2ce4505acbf5743e28c20e","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:04.355Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-bc330c13b9c25f1f8495be73: start","job_status":"start","queue_duration_s":0.475953,"scheduling_latency_s":0.475975}1395gitlab # [ 275.998719] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:04.864Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"bc330c13b9c25f1f8495be73","created_at":"2026-11-13T10:38:04.347Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d183e4ce2f2ce4505acbf5743e28c20e","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:04.355Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-bc330c13b9c25f1f8495be73: done: 0.032566 sec","job_status":"done","queue_duration_s":0.475953,"scheduling_latency_s":0.475975,"redis_calls":10,"redis_duration_s":0.001862,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000346,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001108,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000408,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021219,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00037168,"duration_s":0.032566,"completed_at":"2026-11-13T10:38:04.864Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003777,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1396gitlab # [ 276.042535] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:04.866Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"d104fb14ff1b7d1fa2304117","created_at":"2026-11-13T10:38:04.391Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f991b10608ed6b483fe1c9fd47ba3071","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:04.398Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-d104fb14ff1b7d1fa2304117: start","job_status":"start","queue_duration_s":0.467268,"scheduling_latency_s":0.467301}1397gitlab # [ 276.235457] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.183Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"d104fb14ff1b7d1fa2304117","created_at":"2026-11-13T10:38:04.391Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f991b10608ed6b483fe1c9fd47ba3071","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:04.398Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-d104fb14ff1b7d1fa2304117: done: 0.317272 sec","job_status":"done","queue_duration_s":0.467268,"scheduling_latency_s":0.467301,"redis_calls":562,"redis_duration_s":0.060018,"redis_read_bytes":560,"redis_write_bytes":48760,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000972,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.059046,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.293778,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.317272,"completed_at":"2026-11-13T10:38:05.183Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1398gitlab # [ 276.300425] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.185Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"8d07c5ff2c05a091db4f6765","created_at":"2026-11-13T10:38:04.404Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6b0d8e0a10f7c9a084ea91ac03201361","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:04.413Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-8d07c5ff2c05a091db4f6765: start","job_status":"start","queue_duration_s":0.77207,"scheduling_latency_s":0.772099}1399gitlab # [ 276.315271] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.193Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1400gitlab # [ 276.319032] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.198Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"8d07c5ff2c05a091db4f6765","created_at":"2026-11-13T10:38:04.404Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6b0d8e0a10f7c9a084ea91ac03201361","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:04.413Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-8d07c5ff2c05a091db4f6765: done: 0.013751 sec","job_status":"done","queue_duration_s":0.77207,"scheduling_latency_s":0.772099,"redis_calls":7,"redis_duration_s":0.001179,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000847,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000332,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008409,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013751,"completed_at":"2026-11-13T10:38:05.198Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1401gitlab # [ 276.360881] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.200Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"d07533f14b15d7527b597b8b","created_at":"2026-11-13T10:38:04.443Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"81975a0912ea20e8b5981263fdcb23b4","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:04.470Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-d07533f14b15d7527b597b8b: start","job_status":"start","queue_duration_s":0.729836,"scheduling_latency_s":0.729857}1402gitlab # [ 276.374316] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.225Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"d07533f14b15d7527b597b8b","created_at":"2026-11-13T10:38:04.443Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"81975a0912ea20e8b5981263fdcb23b4","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:04.470Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-d07533f14b15d7527b597b8b: done: 0.024772 sec","job_status":"done","queue_duration_s":0.729836,"scheduling_latency_s":0.729857,"redis_calls":12,"redis_duration_s":0.002248,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000527,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000402,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000939,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00038,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01322,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00027686,"duration_s":0.024772,"completed_at":"2026-11-13T10:38:05.224Z","load_balancing_strategy":"primary","db_duration_s":0.004573,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1403gitlab # [ 276.417535] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.226Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"e701bdb2f90a53830d66a156","created_at":"2026-11-13T10:38:04.479Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e1fd170adf35f112d241e6c618a346bd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-13T10:38:04.486Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-e701bdb2f90a53830d66a156: start","job_status":"start","queue_duration_s":0.740067,"scheduling_latency_s":0.740087}1404gitlab # [ 276.431110] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.249Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"e701bdb2f90a53830d66a156","created_at":"2026-11-13T10:38:04.479Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e1fd170adf35f112d241e6c618a346bd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-13T10:38:04.486Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-e701bdb2f90a53830d66a156: done: 0.023002 sec","job_status":"done","queue_duration_s":0.740067,"scheduling_latency_s":0.740087,"redis_calls":7,"redis_duration_s":0.001516,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001102,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000414,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015915,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023002,"completed_at":"2026-11-13T10:38:05.249Z","load_balancing_strategy":"primary","db_duration_s":0.001771,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1405gitlab # [ 276.471459] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.251Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"6aa98658fb8dd94e248ac747","created_at":"2026-11-13T10:38:04.506Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a8d08e2cecab4ca04a36c3b12770bc4a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-13T10:38:04.514Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-6aa98658fb8dd94e248ac747: start","job_status":"start","queue_duration_s":0.737528,"scheduling_latency_s":0.737557}1406gitlab # [ 276.486128] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.262Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1407gitlab # [ 276.489649] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.270Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"6aa98658fb8dd94e248ac747","created_at":"2026-11-13T10:38:04.506Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a8d08e2cecab4ca04a36c3b12770bc4a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-13T10:38:04.514Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-6aa98658fb8dd94e248ac747: done: 0.018107 sec","job_status":"done","queue_duration_s":0.737528,"scheduling_latency_s":0.737557,"redis_calls":7,"redis_duration_s":0.001504,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001047,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000457,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011426,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018107,"completed_at":"2026-11-13T10:38:05.269Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1408gitlab # [ 276.529984] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.271Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"dba64ff984fdd5bbc82d5407","created_at":"2026-11-13T10:38:04.526Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a11f3dd8e2b7c42f5035ede4f88f88f7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:04.532Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-dba64ff984fdd5bbc82d5407: start","job_status":"start","queue_duration_s":0.739518,"scheduling_latency_s":0.739544}1409gitlab # [ 276.543978] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.309Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"dba64ff984fdd5bbc82d5407","created_at":"2026-11-13T10:38:04.526Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a11f3dd8e2b7c42f5035ede4f88f88f7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:04.532Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-dba64ff984fdd5bbc82d5407: done: 0.037214 sec","job_status":"done","queue_duration_s":0.739518,"scheduling_latency_s":0.739544,"redis_calls":14,"redis_duration_s":0.00319,"redis_read_bytes":16,"redis_write_bytes":4030,"redis_queues_calls":2,"redis_queues_duration_s":0.000405,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002013,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000772,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026982,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.037214,"completed_at":"2026-11-13T10:38:05.308Z","load_balancing_strategy":"primary","db_duration_s":0.003572,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1410gitlab # [ 276.585965] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.310Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"08dab27acaf6d253788f0883","created_at":"2026-11-13T10:38:04.542Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bba68a0cd73b88be62eff4eb397e435f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:04.567Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-08dab27acaf6d253788f0883: start","job_status":"start","queue_duration_s":0.743243,"scheduling_latency_s":0.743265}1411gitlab # [ 276.600945] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.354Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"08dab27acaf6d253788f0883","created_at":"2026-11-13T10:38:04.542Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bba68a0cd73b88be62eff4eb397e435f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:04.567Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-08dab27acaf6d253788f0883: done: 0.04337 sec","job_status":"done","queue_duration_s":0.743243,"scheduling_latency_s":0.743265,"redis_calls":9,"redis_duration_s":0.002351,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001121,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00123,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023079,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00092092,"exclusive_lock_hold_duration_s":0.01786462,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.04337,"completed_at":"2026-11-13T10:38:05.353Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015199,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1412gitlab # [ 276.644972] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.355Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"37dec99b7b01dd0e9e88a3ae","created_at":"2026-11-13T10:38:04.574Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"31a7dedb6304c4fecdec79c8f5053bf9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-13T10:38:04.580Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-37dec99b7b01dd0e9e88a3ae: start","job_status":"start","queue_duration_s":0.775171,"scheduling_latency_s":0.775193}1413gitlab # [ 276.658574] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.365Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1414gitlab # [ 276.662487] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.371Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"37dec99b7b01dd0e9e88a3ae","created_at":"2026-11-13T10:38:04.574Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"31a7dedb6304c4fecdec79c8f5053bf9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-13T10:38:04.580Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-37dec99b7b01dd0e9e88a3ae: done: 0.016049 sec","job_status":"done","queue_duration_s":0.775171,"scheduling_latency_s":0.775193,"redis_calls":7,"redis_duration_s":0.001369,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000987,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000382,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010257,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016049,"completed_at":"2026-11-13T10:38:05.371Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1415gitlab # [ 276.703994] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.373Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"f42a881aaf5697223929eae6","created_at":"2026-11-13T10:38:04.599Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1d0b8972617c5cb7cd3e89e513b4b481","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:04.604Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-f42a881aaf5697223929eae6: start","job_status":"start","queue_duration_s":0.768656,"scheduling_latency_s":0.768676}1416gitlab # [ 276.719024] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.381Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1417gitlab # [ 276.722637] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.387Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"f42a881aaf5697223929eae6","created_at":"2026-11-13T10:38:04.599Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1d0b8972617c5cb7cd3e89e513b4b481","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:04.604Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-f42a881aaf5697223929eae6: done: 0.014181 sec","job_status":"done","queue_duration_s":0.768656,"scheduling_latency_s":0.768676,"redis_calls":7,"redis_duration_s":0.0013059999999999999,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000926,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00038,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00899,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014181,"completed_at":"2026-11-13T10:38:05.387Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1418gitlab # [ 276.764583] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.388Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"4d7782615240ed6335124578","created_at":"2026-11-13T10:38:04.608Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cf23969ccdc90bfe2c053f7b34334be4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:04.611Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-4d7782615240ed6335124578: start","job_status":"start","queue_duration_s":0.77709,"scheduling_latency_s":0.777108}1419gitlab # [ 276.779656] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.401Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"4d7782615240ed6335124578","created_at":"2026-11-13T10:38:04.608Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cf23969ccdc90bfe2c053f7b34334be4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:04.611Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-4d7782615240ed6335124578: done: 0.013286 sec","job_status":"done","queue_duration_s":0.77709,"scheduling_latency_s":0.777108,"redis_calls":7,"redis_duration_s":0.001243,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000878,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000365,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008657,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013286,"completed_at":"2026-11-13T10:38:05.401Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1420gitlab # [ 276.821504] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.403Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"1779222d8f5faab20c9a48e1","created_at":"2026-11-13T10:38:04.628Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"481a5517e264f5182feb53c357bcd8dc","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:04.640Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-1779222d8f5faab20c9a48e1: start","job_status":"start","queue_duration_s":0.763115,"scheduling_latency_s":0.763132}1421gitlab # [ 276.836368] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.444Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"1779222d8f5faab20c9a48e1","created_at":"2026-11-13T10:38:04.628Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"481a5517e264f5182feb53c357bcd8dc","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:04.640Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-1779222d8f5faab20c9a48e1: done: 0.041199 sec","job_status":"done","queue_duration_s":0.763115,"scheduling_latency_s":0.763132,"redis_calls":12,"redis_duration_s":0.002244,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000407,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00096,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000877,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013806,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000514,"exclusive_lock_hold_duration_s":0.005465220000000001,"duration_s":0.041199,"completed_at":"2026-11-13T10:38:05.444Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002973,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1422gitlab # [ 276.879806] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.445Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"02972c5ad2499cb9bd33991f","created_at":"2026-11-13T10:38:04.663Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"afbe536a051ebc3d641078f6108523fe","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:04.666Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-02972c5ad2499cb9bd33991f: start","job_status":"start","queue_duration_s":0.77896,"scheduling_latency_s":0.778978}1423gitlab # [ 276.895473] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.454Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1424gitlab # [ 276.898927] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.460Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"02972c5ad2499cb9bd33991f","created_at":"2026-11-13T10:38:04.663Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"afbe536a051ebc3d641078f6108523fe","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:04.666Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-02972c5ad2499cb9bd33991f: done: 0.014543 sec","job_status":"done","queue_duration_s":0.77896,"scheduling_latency_s":0.778978,"redis_calls":7,"redis_duration_s":0.002149,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001807,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000342,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008501,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014543,"completed_at":"2026-11-13T10:38:05.460Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1425gitlab # [ 276.940442] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.461Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"a362b0814291a8a93000488b","created_at":"2026-11-13T10:38:04.686Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f9663272c67dd041fdcad4b7d5e347c0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:04.693Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-a362b0814291a8a93000488b: start","job_status":"start","queue_duration_s":0.767911,"scheduling_latency_s":0.767927}1426gitlab # [ 276.954817] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.673Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"a362b0814291a8a93000488b","created_at":"2026-11-13T10:38:04.686Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f9663272c67dd041fdcad4b7d5e347c0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:04.693Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-a362b0814291a8a93000488b: done: 0.212078 sec","job_status":"done","queue_duration_s":0.767911,"scheduling_latency_s":0.767927,"redis_calls":11,"redis_duration_s":0.0022890000000000002,"redis_read_bytes":9,"redis_write_bytes":1262,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.0004,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":275,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000937,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000952,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":130,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.03,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.174125,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00056448,"exclusive_lock_hold_duration_s":0.19369972,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.212078,"completed_at":"2026-11-13T10:38:05.673Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.044318,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1427gitlab # [ 277.000239] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.674Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"21eacf450d30f59836cba3db","created_at":"2026-11-13T10:38:04.703Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c3b5ad2d29d11a5dece9849af205c19b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:04.708Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-21eacf450d30f59836cba3db: start","job_status":"start","queue_duration_s":0.966787,"scheduling_latency_s":0.966807}1428gitlab # [ 277.015122] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.683Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1429gitlab # [ 277.019155] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.688Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"21eacf450d30f59836cba3db","created_at":"2026-11-13T10:38:04.703Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c3b5ad2d29d11a5dece9849af205c19b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:04.708Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-21eacf450d30f59836cba3db: done: 0.013882 sec","job_status":"done","queue_duration_s":0.966787,"scheduling_latency_s":0.966807,"redis_calls":7,"redis_duration_s":0.001215,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000856,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000359,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008921,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013882,"completed_at":"2026-11-13T10:38:05.688Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1430gitlab # [ 277.060880] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.690Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"fd6b7105f83e5f6280b9878b","created_at":"2026-11-13T10:38:04.728Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:04.734Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-fd6b7105f83e5f6280b9878b: start","job_status":"start","queue_duration_s":0.955841,"scheduling_latency_s":0.955862}1431gitlab # [ 277.074819] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.791Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"fd6b7105f83e5f6280b9878b","created_at":"2026-11-13T10:38:04.728Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:04.734Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-fd6b7105f83e5f6280b9878b: done: 0.101754 sec","job_status":"done","queue_duration_s":0.955841,"scheduling_latency_s":0.955862,"redis_calls":60,"redis_duration_s":0.011493,"redis_read_bytes":108,"redis_write_bytes":41109,"redis_queues_calls":2,"redis_queues_duration_s":0.000552,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.010408,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000533,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.089993,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.101754,"completed_at":"2026-11-13T10:38:05.791Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1432gitlab # [ 277.117739] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.793Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"6af16a7b699c31f83bf49e36","created_at":"2026-11-13T10:38:04.740Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"db8f217713119edb47cf849d6c76dd21","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:04.744Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-6af16a7b699c31f83bf49e36: start","job_status":"start","queue_duration_s":1.048368,"scheduling_latency_s":1.048388}1433gitlab # [ 277.132197] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.809Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"6af16a7b699c31f83bf49e36","created_at":"2026-11-13T10:38:04.740Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-d2f5bc27122747aa","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"db8f217713119edb47cf849d6c76dd21","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:04.744Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-6af16a7b699c31f83bf49e36: done: 0.015905 sec","job_status":"done","queue_duration_s":1.048368,"scheduling_latency_s":1.048388,"redis_calls":7,"redis_duration_s":0.001312,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000959,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000353,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010163,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015905,"completed_at":"2026-11-13T10:38:05.809Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001165,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1434gitlab # [ 277.173137] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.810Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"04bb23dd265ec7cfe78315d7","created_at":"2026-11-13T10:38:05.288Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-bd348f447e1b4d81","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a11f3dd8e2b7c42f5035ede4f88f88f7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.301Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-04bb23dd265ec7cfe78315d7: start","job_status":"start","queue_duration_s":0.509131,"scheduling_latency_s":0.509147}1435gitlab # [ 277.185822] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.822Z","message":"No existing merge request to be cleaned up."}1436gitlab # [ 277.187530] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.830Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"04bb23dd265ec7cfe78315d7","created_at":"2026-11-13T10:38:05.288Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-bd348f447e1b4d81","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a11f3dd8e2b7c42f5035ede4f88f88f7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.301Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-04bb23dd265ec7cfe78315d7: done: 0.01993 sec","job_status":"done","queue_duration_s":0.509131,"scheduling_latency_s":0.509147,"redis_calls":10,"redis_duration_s":0.001853,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000785,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001068,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013805,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01993,"completed_at":"2026-11-13T10:38:05.830Z","load_balancing_strategy":"primary","db_duration_s":0.001927,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1437gitlab # [ 277.227136] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.831Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"cd8b26eeef0324632fc39ea9","created_at":"2026-11-13T10:38:05.288Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-bd348f447e1b4d81","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a11f3dd8e2b7c42f5035ede4f88f88f7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.301Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-cd8b26eeef0324632fc39ea9: start","job_status":"start","queue_duration_s":0.530364,"scheduling_latency_s":0.530381}1438gitlab # [ 277.240572] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.843Z","message":"No existing merge request to be cleaned up."}1439gitlab # [ 277.242299] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.851Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"cd8b26eeef0324632fc39ea9","created_at":"2026-11-13T10:38:05.288Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-bd348f447e1b4d81","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a11f3dd8e2b7c42f5035ede4f88f88f7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.301Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-cd8b26eeef0324632fc39ea9: done: 0.019702 sec","job_status":"done","queue_duration_s":0.530364,"scheduling_latency_s":0.530381,"redis_calls":10,"redis_duration_s":0.001761,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000726,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001035,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012699,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019702,"completed_at":"2026-11-13T10:38:05.851Z","load_balancing_strategy":"primary","db_duration_s":0.001485,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1440gitlab # [ 277.281570] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.852Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"23b3cdd65e896ae3fd0d26be","created_at":"2026-11-13T10:38:05.288Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-bd348f447e1b4d81","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a11f3dd8e2b7c42f5035ede4f88f88f7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.301Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-23b3cdd65e896ae3fd0d26be: start","job_status":"start","queue_duration_s":0.551269,"scheduling_latency_s":0.551286}1441gitlab # [ 277.295151] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.864Z","message":"No existing merge request to be cleaned up."}1442gitlab # [ 277.296953] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.871Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"23b3cdd65e896ae3fd0d26be","created_at":"2026-11-13T10:38:05.288Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-bd348f447e1b4d81","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a11f3dd8e2b7c42f5035ede4f88f88f7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.301Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-23b3cdd65e896ae3fd0d26be: done: 0.018823 sec","job_status":"done","queue_duration_s":0.551269,"scheduling_latency_s":0.551286,"redis_calls":10,"redis_duration_s":0.0018499999999999999,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000774,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001076,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013231,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018823,"completed_at":"2026-11-13T10:38:05.871Z","load_balancing_strategy":"primary","db_duration_s":0.001528,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1443gitlab # [ 277.336625] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.873Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"3dfd7d3ea445d9e281125f98","created_at":"2026-11-13T10:38:05.288Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-bd348f447e1b4d81","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a11f3dd8e2b7c42f5035ede4f88f88f7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.301Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-3dfd7d3ea445d9e281125f98: start","job_status":"start","queue_duration_s":0.572218,"scheduling_latency_s":0.572236}1444gitlab # [ 277.349092] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.883Z","message":"No existing merge request to be cleaned up."}1445gitlab # [ 277.350816] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.891Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"3dfd7d3ea445d9e281125f98","created_at":"2026-11-13T10:38:05.288Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-bd348f447e1b4d81","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a11f3dd8e2b7c42f5035ede4f88f88f7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.301Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-3dfd7d3ea445d9e281125f98: done: 0.018364 sec","job_status":"done","queue_duration_s":0.572218,"scheduling_latency_s":0.572236,"redis_calls":10,"redis_duration_s":0.001941,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000713,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001228,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012593,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018364,"completed_at":"2026-11-13T10:38:05.891Z","load_balancing_strategy":"primary","db_duration_s":0.001515,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1446gitlab # [ 277.390136] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.893Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5ab1c525fa5752c9710ca9ba","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5ab1c525fa5752c9710ca9ba: start","job_status":"start","queue_duration_s":0.10873,"scheduling_latency_s":0.108746}1447gitlab # [ 277.402852] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.908Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5ab1c525fa5752c9710ca9ba","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5ab1c525fa5752c9710ca9ba: done: 0.015866 sec","job_status":"done","queue_duration_s":0.10873,"scheduling_latency_s":0.108746,"redis_calls":13,"redis_duration_s":0.002091,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000663,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001428,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01091,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015866,"completed_at":"2026-11-13T10:38:05.908Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1448gitlab # [ 277.442786] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.910Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d5940360e9df73ecba18a9db","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d5940360e9df73ecba18a9db: start","job_status":"start","queue_duration_s":0.125752,"scheduling_latency_s":0.125769}1449gitlab # [ 277.455651] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.925Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d5940360e9df73ecba18a9db","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d5940360e9df73ecba18a9db: done: 0.015331 sec","job_status":"done","queue_duration_s":0.125752,"scheduling_latency_s":0.125769,"redis_calls":13,"redis_duration_s":0.001941,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000648,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001293,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009332,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015331,"completed_at":"2026-11-13T10:38:05.925Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1450gitlab # [ 277.496900] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.927Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4a1a5ab2e84fab2a84264665","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4a1a5ab2e84fab2a84264665: start","job_status":"start","queue_duration_s":0.143555,"scheduling_latency_s":0.143573}1451gitlab # [ 277.509728] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.943Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4a1a5ab2e84fab2a84264665","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4a1a5ab2e84fab2a84264665: done: 0.015649 sec","job_status":"done","queue_duration_s":0.143555,"scheduling_latency_s":0.143573,"redis_calls":13,"redis_duration_s":0.001882,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000704,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001178,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010834,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015649,"completed_at":"2026-11-13T10:38:05.943Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1452gitlab # [ 277.550024] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.944Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"86fed72f60b14ce45721eb1a","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-86fed72f60b14ce45721eb1a: start","job_status":"start","queue_duration_s":0.160342,"scheduling_latency_s":0.160359}1453gitlab # [ 277.562899] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.960Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"86fed72f60b14ce45721eb1a","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-86fed72f60b14ce45721eb1a: done: 0.0155 sec","job_status":"done","queue_duration_s":0.160342,"scheduling_latency_s":0.160359,"redis_calls":13,"redis_duration_s":0.0017239999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000602,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001122,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010948,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0155,"completed_at":"2026-11-13T10:38:05.960Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1454gitlab # [ 277.603545] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.961Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"281cc89ee5d252f86ac37a64","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-281cc89ee5d252f86ac37a64: start","job_status":"start","queue_duration_s":0.176964,"scheduling_latency_s":0.176982}1455gitlab # [ 277.616741] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.976Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"281cc89ee5d252f86ac37a64","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-281cc89ee5d252f86ac37a64: done: 0.015536 sec","job_status":"done","queue_duration_s":0.176964,"scheduling_latency_s":0.176982,"redis_calls":13,"redis_duration_s":0.0017230000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000613,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00111,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010995,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015536,"completed_at":"2026-11-13T10:38:05.976Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1456gitlab # [ 277.657206] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.978Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"df6ba10cc8a633587cdcf08b","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-df6ba10cc8a633587cdcf08b: start","job_status":"start","queue_duration_s":0.193697,"scheduling_latency_s":0.193715}1457gitlab # [ 277.670152] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.994Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"df6ba10cc8a633587cdcf08b","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-df6ba10cc8a633587cdcf08b: done: 0.016853 sec","job_status":"done","queue_duration_s":0.193697,"scheduling_latency_s":0.193715,"redis_calls":13,"redis_duration_s":0.0016510000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000579,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001072,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009861,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016853,"completed_at":"2026-11-13T10:38:05.994Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1458gitlab # [ 277.711798] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:05.996Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea7332ee3ca302f162132d5e","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea7332ee3ca302f162132d5e: start","job_status":"start","queue_duration_s":0.211781,"scheduling_latency_s":0.211798}1459gitlab # [ 277.724923] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.011Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea7332ee3ca302f162132d5e","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea7332ee3ca302f162132d5e: done: 0.015055 sec","job_status":"done","queue_duration_s":0.211781,"scheduling_latency_s":0.211798,"redis_calls":13,"redis_duration_s":0.00155,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000564,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000986,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010741,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015055,"completed_at":"2026-11-13T10:38:06.011Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1460gitlab # [ 277.765379] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.012Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3d7cb729ec2a39d2e65d1f66","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3d7cb729ec2a39d2e65d1f66: start","job_status":"start","queue_duration_s":0.227959,"scheduling_latency_s":0.227977}1461gitlab # [ 277.778305] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.026Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3d7cb729ec2a39d2e65d1f66","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3d7cb729ec2a39d2e65d1f66: done: 0.014642 sec","job_status":"done","queue_duration_s":0.227959,"scheduling_latency_s":0.227977,"redis_calls":13,"redis_duration_s":0.00151,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000546,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000964,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010392,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014642,"completed_at":"2026-11-13T10:38:06.026Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1462gitlab # [ 277.819073] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.028Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"326a1e4ce52fc33743d68c4b","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-326a1e4ce52fc33743d68c4b: start","job_status":"start","queue_duration_s":0.243765,"scheduling_latency_s":0.243782}1463gitlab # [ 277.832039] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.042Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"326a1e4ce52fc33743d68c4b","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-326a1e4ce52fc33743d68c4b: done: 0.014774 sec","job_status":"done","queue_duration_s":0.243765,"scheduling_latency_s":0.243782,"redis_calls":13,"redis_duration_s":0.001529,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000571,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000958,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009798,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014774,"completed_at":"2026-11-13T10:38:06.042Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1464gitlab # [ 277.872763] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.044Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"066d78c4b17fce97bc453581","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-066d78c4b17fce97bc453581: start","job_status":"start","queue_duration_s":0.260191,"scheduling_latency_s":0.260209}1465gitlab # [ 277.885452] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.059Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"066d78c4b17fce97bc453581","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-066d78c4b17fce97bc453581: done: 0.014813 sec","job_status":"done","queue_duration_s":0.260191,"scheduling_latency_s":0.260209,"redis_calls":13,"redis_duration_s":0.001513,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000567,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000946,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010515,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014813,"completed_at":"2026-11-13T10:38:06.059Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1466gitlab # [ 277.927337] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.060Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6d0710e8a5073b48a376d989","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6d0710e8a5073b48a376d989: start","job_status":"start","queue_duration_s":0.276105,"scheduling_latency_s":0.276123}1467gitlab # [ 277.940371] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.075Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6d0710e8a5073b48a376d989","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6d0710e8a5073b48a376d989: done: 0.014633 sec","job_status":"done","queue_duration_s":0.276105,"scheduling_latency_s":0.276123,"redis_calls":13,"redis_duration_s":0.0014810000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000518,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000963,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010438,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014633,"completed_at":"2026-11-13T10:38:06.074Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1468gitlab # [ 277.981344] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.076Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4a76ba578e0292621d7e72e0","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4a76ba578e0292621d7e72e0: start","job_status":"start","queue_duration_s":0.291885,"scheduling_latency_s":0.291903}1469gitlab # [ 277.994317] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.091Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4a76ba578e0292621d7e72e0","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4a76ba578e0292621d7e72e0: done: 0.015354 sec","job_status":"done","queue_duration_s":0.291885,"scheduling_latency_s":0.291903,"redis_calls":13,"redis_duration_s":0.00181,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000563,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001247,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010788,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015354,"completed_at":"2026-11-13T10:38:06.091Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1470gitlab # [ 278.035224] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.093Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"99bc91263187d51184beb469","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-99bc91263187d51184beb469: start","job_status":"start","queue_duration_s":0.309001,"scheduling_latency_s":0.309021}1471gitlab # [ 278.048279] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.108Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"99bc91263187d51184beb469","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-99bc91263187d51184beb469: done: 0.014862 sec","job_status":"done","queue_duration_s":0.309001,"scheduling_latency_s":0.309021,"redis_calls":13,"redis_duration_s":0.001482,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00053,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000952,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010486,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014862,"completed_at":"2026-11-13T10:38:06.108Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1472gitlab # [ 278.088981] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.109Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5d6cf3dfa492e7400b845f3b","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5d6cf3dfa492e7400b845f3b: start","job_status":"start","queue_duration_s":0.324944,"scheduling_latency_s":0.324962}1473gitlab # [ 278.101913] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.124Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5d6cf3dfa492e7400b845f3b","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5d6cf3dfa492e7400b845f3b: done: 0.01503 sec","job_status":"done","queue_duration_s":0.324944,"scheduling_latency_s":0.324962,"redis_calls":13,"redis_duration_s":0.002171,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000517,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001654,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010664,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01503,"completed_at":"2026-11-13T10:38:06.124Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1474gitlab # [ 278.143417] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.125Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d82e0f607a13943e49a081a6","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d82e0f607a13943e49a081a6: start","job_status":"start","queue_duration_s":0.341051,"scheduling_latency_s":0.341068}1475gitlab # [ 278.156494] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.140Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d82e0f607a13943e49a081a6","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d82e0f607a13943e49a081a6: done: 0.015063 sec","job_status":"done","queue_duration_s":0.341051,"scheduling_latency_s":0.341068,"redis_calls":13,"redis_duration_s":0.001543,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000563,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00098,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010683,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015063,"completed_at":"2026-11-13T10:38:06.140Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1476gitlab # [ 278.196901] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.141Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c1816682102fc70d133aadb3","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c1816682102fc70d133aadb3: start","job_status":"start","queue_duration_s":0.35726,"scheduling_latency_s":0.357278}1477gitlab # [ 278.209621] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.157Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c1816682102fc70d133aadb3","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c1816682102fc70d133aadb3: done: 0.016057 sec","job_status":"done","queue_duration_s":0.35726,"scheduling_latency_s":0.357278,"redis_calls":13,"redis_duration_s":0.001681,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000582,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001099,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011003,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016057,"completed_at":"2026-11-13T10:38:06.157Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1478gitlab # [ 278.250971] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.159Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"181513697c787fec0a007e85","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-181513697c787fec0a007e85: start","job_status":"start","queue_duration_s":0.374903,"scheduling_latency_s":0.374922}1479gitlab # [ 278.264002] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.175Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"181513697c787fec0a007e85","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-181513697c787fec0a007e85: done: 0.016019 sec","job_status":"done","queue_duration_s":0.374903,"scheduling_latency_s":0.374922,"redis_calls":13,"redis_duration_s":0.001945,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000575,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00137,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011065,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016019,"completed_at":"2026-11-13T10:38:06.175Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1480gitlab # [ 278.305388] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.176Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cce0d207182361f93074b88f","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cce0d207182361f93074b88f: start","job_status":"start","queue_duration_s":0.392035,"scheduling_latency_s":0.392053}1481gitlab # [ 278.319351] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.191Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cce0d207182361f93074b88f","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cce0d207182361f93074b88f: done: 0.015204 sec","job_status":"done","queue_duration_s":0.392035,"scheduling_latency_s":0.392053,"redis_calls":13,"redis_duration_s":0.001728,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000754,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000974,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010656,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015204,"completed_at":"2026-11-13T10:38:06.191Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1482gitlab # [ 278.360307] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.192Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b5e5fd89f80730d9e1cb4b7","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b5e5fd89f80730d9e1cb4b7: start","job_status":"start","queue_duration_s":0.408375,"scheduling_latency_s":0.408394}1483gitlab # [ 278.373242] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.207Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b5e5fd89f80730d9e1cb4b7","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b5e5fd89f80730d9e1cb4b7: done: 0.014871 sec","job_status":"done","queue_duration_s":0.408375,"scheduling_latency_s":0.408394,"redis_calls":13,"redis_duration_s":0.0015370000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000543,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000994,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010028,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014871,"completed_at":"2026-11-13T10:38:06.207Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1484gitlab # [ 278.413924] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.209Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"50a5cf7789012bef4145a286","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-50a5cf7789012bef4145a286: start","job_status":"start","queue_duration_s":0.424995,"scheduling_latency_s":0.425014}1485gitlab # [ 278.426936] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.224Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"50a5cf7789012bef4145a286","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-50a5cf7789012bef4145a286: done: 0.014722 sec","job_status":"done","queue_duration_s":0.424995,"scheduling_latency_s":0.425014,"redis_calls":13,"redis_duration_s":0.001487,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000962,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010231,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014722,"completed_at":"2026-11-13T10:38:06.223Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1486gitlab # [ 278.467957] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.225Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"10928fd39880b759961fe6e8","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-10928fd39880b759961fe6e8: start","job_status":"start","queue_duration_s":0.440837,"scheduling_latency_s":0.440855}1487gitlab # [ 278.480890] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.240Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"10928fd39880b759961fe6e8","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-10928fd39880b759961fe6e8: done: 0.014944 sec","job_status":"done","queue_duration_s":0.440837,"scheduling_latency_s":0.440855,"redis_calls":13,"redis_duration_s":0.001555,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000543,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001012,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010576,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014944,"completed_at":"2026-11-13T10:38:06.240Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1488gitlab # [ 278.522524] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.241Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad48b02b2a530c6283b91c17","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad48b02b2a530c6283b91c17: start","job_status":"start","queue_duration_s":0.456871,"scheduling_latency_s":0.456889}1489gitlab # [ 278.535394] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.256Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad48b02b2a530c6283b91c17","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad48b02b2a530c6283b91c17: done: 0.015462 sec","job_status":"done","queue_duration_s":0.456871,"scheduling_latency_s":0.456889,"redis_calls":13,"redis_duration_s":0.001577,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000578,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000999,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010883,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015462,"completed_at":"2026-11-13T10:38:06.256Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1490gitlab # [ 278.576253] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.257Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5d2413935ee1b8373a231dff","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5d2413935ee1b8373a231dff: start","job_status":"start","queue_duration_s":0.47346,"scheduling_latency_s":0.473478}1491gitlab # [ 278.589125] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.272Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5d2413935ee1b8373a231dff","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5d2413935ee1b8373a231dff: done: 0.014545 sec","job_status":"done","queue_duration_s":0.47346,"scheduling_latency_s":0.473478,"redis_calls":13,"redis_duration_s":0.00149,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00052,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00097,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009734,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014545,"completed_at":"2026-11-13T10:38:06.272Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1492gitlab # [ 278.629668] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.273Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e4dcdf969dbcfe8e42f5390","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e4dcdf969dbcfe8e42f5390: start","job_status":"start","queue_duration_s":0.489193,"scheduling_latency_s":0.489211}1493gitlab # [ 278.642579] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.288Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e4dcdf969dbcfe8e42f5390","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e4dcdf969dbcfe8e42f5390: done: 0.01484 sec","job_status":"done","queue_duration_s":0.489193,"scheduling_latency_s":0.489211,"redis_calls":13,"redis_duration_s":0.001801,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000561,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00124,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010191,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01484,"completed_at":"2026-11-13T10:38:06.288Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1494gitlab # [ 278.683104] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.289Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae6cc240985cfc9b20635398","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae6cc240985cfc9b20635398: start","job_status":"start","queue_duration_s":0.50521,"scheduling_latency_s":0.505229}1495gitlab # [ 278.696120] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.304Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae6cc240985cfc9b20635398","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae6cc240985cfc9b20635398: done: 0.014728 sec","job_status":"done","queue_duration_s":0.50521,"scheduling_latency_s":0.505229,"redis_calls":13,"redis_duration_s":0.001582,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000594,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000988,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009963,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014728,"completed_at":"2026-11-13T10:38:06.304Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1496gitlab # [ 278.737518] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.305Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"174b7428295de6714d383fd1","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-174b7428295de6714d383fd1: start","job_status":"start","queue_duration_s":0.521081,"scheduling_latency_s":0.521118}1497gitlab # [ 278.750566] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.319Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"174b7428295de6714d383fd1","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-174b7428295de6714d383fd1: done: 0.01429 sec","job_status":"done","queue_duration_s":0.521081,"scheduling_latency_s":0.521118,"redis_calls":13,"redis_duration_s":0.001546,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000571,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000975,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009856,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01429,"completed_at":"2026-11-13T10:38:06.319Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1498gitlab # [ 278.790890] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.320Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6f1f23a8fddb5d84d2dfffd","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6f1f23a8fddb5d84d2dfffd: start","job_status":"start","queue_duration_s":0.536617,"scheduling_latency_s":0.536636}1499gitlab # [ 278.803866] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.335Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6f1f23a8fddb5d84d2dfffd","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6f1f23a8fddb5d84d2dfffd: done: 0.014669 sec","job_status":"done","queue_duration_s":0.536617,"scheduling_latency_s":0.536636,"redis_calls":13,"redis_duration_s":0.001562,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000589,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000973,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010339,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014669,"completed_at":"2026-11-13T10:38:06.335Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1500gitlab # [ 278.844778] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.336Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f12b6cac131d7e07b502fb30","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f12b6cac131d7e07b502fb30: start","job_status":"start","queue_duration_s":0.552396,"scheduling_latency_s":0.552414}1501gitlab # [ 278.857706] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.351Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f12b6cac131d7e07b502fb30","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f12b6cac131d7e07b502fb30: done: 0.014483 sec","job_status":"done","queue_duration_s":0.552396,"scheduling_latency_s":0.552414,"redis_calls":13,"redis_duration_s":0.001535,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001003,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01007,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014483,"completed_at":"2026-11-13T10:38:06.351Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1502gitlab # [ 278.898187] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.352Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e15a7bd131f9844df684adb","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e15a7bd131f9844df684adb: start","job_status":"start","queue_duration_s":0.568069,"scheduling_latency_s":0.568087}1503gitlab # [ 278.911223] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.366Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e15a7bd131f9844df684adb","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e15a7bd131f9844df684adb: done: 0.014353 sec","job_status":"done","queue_duration_s":0.568069,"scheduling_latency_s":0.568087,"redis_calls":13,"redis_duration_s":0.001486,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000958,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014353,"completed_at":"2026-11-13T10:38:06.366Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1504gitlab # [ 278.953065] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.367Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"52ed76b9f313323ad1805b14","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-52ed76b9f313323ad1805b14: start","job_status":"start","queue_duration_s":0.583616,"scheduling_latency_s":0.583634}1505gitlab # [ 278.965803] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.382Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"52ed76b9f313323ad1805b14","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-52ed76b9f313323ad1805b14: done: 0.014147 sec","job_status":"done","queue_duration_s":0.583616,"scheduling_latency_s":0.583634,"redis_calls":13,"redis_duration_s":0.001575,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000546,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001029,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009857,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014147,"completed_at":"2026-11-13T10:38:06.381Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1506gitlab # [ 279.006305] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.383Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b088679b316c6376da94237c","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b088679b316c6376da94237c: start","job_status":"start","queue_duration_s":0.599179,"scheduling_latency_s":0.599199}1507gitlab # [ 279.019319] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.398Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b088679b316c6376da94237c","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b088679b316c6376da94237c: done: 0.014812 sec","job_status":"done","queue_duration_s":0.599179,"scheduling_latency_s":0.599199,"redis_calls":13,"redis_duration_s":0.001645,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000547,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001098,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010279,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014812,"completed_at":"2026-11-13T10:38:06.398Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1508gitlab # [ 279.059987] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.399Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68859ff759b36a3062eb0e50","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68859ff759b36a3062eb0e50: start","job_status":"start","queue_duration_s":0.615133,"scheduling_latency_s":0.615151}1509gitlab # [ 279.072843] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.433Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68859ff759b36a3062eb0e50","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68859ff759b36a3062eb0e50: done: 0.033765 sec","job_status":"done","queue_duration_s":0.615133,"scheduling_latency_s":0.615151,"redis_calls":13,"redis_duration_s":0.0031349999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001207,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001928,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025815,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.033765,"completed_at":"2026-11-13T10:38:06.433Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1510gitlab # [ 279.113384] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.436Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9679b0f5d88aca106a3be130","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9679b0f5d88aca106a3be130: start","job_status":"start","queue_duration_s":0.65213,"scheduling_latency_s":0.652181}1511gitlab # [ 279.127147] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.476Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9679b0f5d88aca106a3be130","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9679b0f5d88aca106a3be130: done: 0.040288 sec","job_status":"done","queue_duration_s":0.65213,"scheduling_latency_s":0.652181,"redis_calls":13,"redis_duration_s":0.003558,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001163,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002395,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028323,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.040288,"completed_at":"2026-11-13T10:38:06.476Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1512gitlab # [ 279.167569] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.479Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"60b070972b7bcaf08bf4ba11","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-60b070972b7bcaf08bf4ba11: start","job_status":"start","queue_duration_s":0.694901,"scheduling_latency_s":0.694969}1513gitlab # [ 279.180431] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.517Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"60b070972b7bcaf08bf4ba11","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-60b070972b7bcaf08bf4ba11: done: 0.038557 sec","job_status":"done","queue_duration_s":0.694901,"scheduling_latency_s":0.694969,"redis_calls":13,"redis_duration_s":0.0034990000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0011,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002399,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028006,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.038557,"completed_at":"2026-11-13T10:38:06.517Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1514gitlab # [ 279.220905] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.520Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9390784812c7ff51790f9873","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9390784812c7ff51790f9873: start","job_status":"start","queue_duration_s":0.736436,"scheduling_latency_s":0.736486}1515gitlab # [ 279.233734] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.559Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9390784812c7ff51790f9873","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9390784812c7ff51790f9873: done: 0.038873 sec","job_status":"done","queue_duration_s":0.736436,"scheduling_latency_s":0.736486,"redis_calls":13,"redis_duration_s":0.003295,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001075,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00222,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028788,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.038873,"completed_at":"2026-11-13T10:38:06.559Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1516gitlab # [ 279.274276] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.562Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"598ffd7ef4196dd8919665a8","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-598ffd7ef4196dd8919665a8: start","job_status":"start","queue_duration_s":0.778108,"scheduling_latency_s":0.778148}1517gitlab # [ 279.287196] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.602Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"598ffd7ef4196dd8919665a8","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-598ffd7ef4196dd8919665a8: done: 0.040499 sec","job_status":"done","queue_duration_s":0.778108,"scheduling_latency_s":0.778148,"redis_calls":13,"redis_duration_s":0.003608,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001142,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002466,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029852,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.040499,"completed_at":"2026-11-13T10:38:06.602Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1518gitlab # [ 279.329181] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.606Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"adbdfa97a8eaa742066a1f9e","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-adbdfa97a8eaa742066a1f9e: start","job_status":"start","queue_duration_s":0.821769,"scheduling_latency_s":0.821822}1519gitlab # [ 279.342242] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.640Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"adbdfa97a8eaa742066a1f9e","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-adbdfa97a8eaa742066a1f9e: done: 0.034375 sec","job_status":"done","queue_duration_s":0.821769,"scheduling_latency_s":0.821822,"redis_calls":13,"redis_duration_s":0.003173,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001035,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002138,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025199,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.034375,"completed_at":"2026-11-13T10:38:06.640Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1520gitlab # [ 279.382864] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.643Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"269da491e1f3c6ca7b463639","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-269da491e1f3c6ca7b463639: start","job_status":"start","queue_duration_s":0.858712,"scheduling_latency_s":0.858739}1521gitlab # [ 279.395789] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.682Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"269da491e1f3c6ca7b463639","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-269da491e1f3c6ca7b463639: done: 0.039465 sec","job_status":"done","queue_duration_s":0.858712,"scheduling_latency_s":0.858739,"redis_calls":13,"redis_duration_s":0.003418,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001129,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002289,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028727,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.039465,"completed_at":"2026-11-13T10:38:06.682Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1522gitlab # [ 279.436090] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.685Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a087f003763192d3a92ef3a5","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a087f003763192d3a92ef3a5: start","job_status":"start","queue_duration_s":0.901382,"scheduling_latency_s":0.90144}1523gitlab # [ 279.448665] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.725Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a087f003763192d3a92ef3a5","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a087f003763192d3a92ef3a5: done: 0.039684 sec","job_status":"done","queue_duration_s":0.901382,"scheduling_latency_s":0.90144,"redis_calls":13,"redis_duration_s":0.003266,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001034,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002232,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028525,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.039684,"completed_at":"2026-11-13T10:38:06.725Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1524gitlab # [ 279.488733] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.728Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"60b55fb07337f0f65cee1192","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-60b55fb07337f0f65cee1192: start","job_status":"start","queue_duration_s":0.943685,"scheduling_latency_s":0.943724}1525gitlab # [ 279.501484] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.771Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"60b55fb07337f0f65cee1192","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-60b55fb07337f0f65cee1192: done: 0.043479 sec","job_status":"done","queue_duration_s":0.943685,"scheduling_latency_s":0.943724,"redis_calls":13,"redis_duration_s":0.003995,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0012,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002795,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031888,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.043479,"completed_at":"2026-11-13T10:38:06.771Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1526gitlab # [ 279.542445] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.774Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b6cf15eee0edf3ef270565d","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b6cf15eee0edf3ef270565d: start","job_status":"start","queue_duration_s":0.989906,"scheduling_latency_s":0.98997}1527gitlab # [ 279.555152] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.817Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b6cf15eee0edf3ef270565d","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b6cf15eee0edf3ef270565d: done: 0.043637 sec","job_status":"done","queue_duration_s":0.989906,"scheduling_latency_s":0.98997,"redis_calls":13,"redis_duration_s":0.003636,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001203,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002433,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03212,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.043637,"completed_at":"2026-11-13T10:38:06.817Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1528gitlab # [ 279.595446] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.820Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aa3bb89f6f62f4a6db2a5240","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aa3bb89f6f62f4a6db2a5240: start","job_status":"start","queue_duration_s":1.035913,"scheduling_latency_s":1.035943}1529gitlab # [ 279.608266] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.863Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aa3bb89f6f62f4a6db2a5240","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aa3bb89f6f62f4a6db2a5240: done: 0.042942 sec","job_status":"done","queue_duration_s":1.035913,"scheduling_latency_s":1.035943,"redis_calls":13,"redis_duration_s":0.003945000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001235,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00271,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032013,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.042942,"completed_at":"2026-11-13T10:38:06.863Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1530gitlab # [ 279.648529] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.866Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b741e381333bfc7037862447","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b741e381333bfc7037862447: start","job_status":"start","queue_duration_s":1.082038,"scheduling_latency_s":1.082099}1531gitlab # [ 279.661062] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.907Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b741e381333bfc7037862447","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b741e381333bfc7037862447: done: 0.041136 sec","job_status":"done","queue_duration_s":1.082038,"scheduling_latency_s":1.082099,"redis_calls":13,"redis_duration_s":0.003611,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001265,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002346,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029935,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.041136,"completed_at":"2026-11-13T10:38:06.907Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1532gitlab # [ 279.700833] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.910Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b1aec9cba23102abcc091cd4","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b1aec9cba23102abcc091cd4: start","job_status":"start","queue_duration_s":1.125773,"scheduling_latency_s":1.125801}1533gitlab # [ 279.713417] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.953Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b1aec9cba23102abcc091cd4","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b1aec9cba23102abcc091cd4: done: 0.043371 sec","job_status":"done","queue_duration_s":1.125773,"scheduling_latency_s":1.125801,"redis_calls":13,"redis_duration_s":0.003765,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001279,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002486,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033041,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.043371,"completed_at":"2026-11-13T10:38:06.953Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1534gitlab # [ 279.753916] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.956Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"527d2232f21824372700863e","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-527d2232f21824372700863e: start","job_status":"start","queue_duration_s":1.172487,"scheduling_latency_s":1.172535}1535gitlab # [ 279.766815] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.997Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"527d2232f21824372700863e","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-527d2232f21824372700863e: done: 0.040359 sec","job_status":"done","queue_duration_s":1.172487,"scheduling_latency_s":1.172535,"redis_calls":13,"redis_duration_s":0.0036769999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001132,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002545,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028721,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.040359,"completed_at":"2026-11-13T10:38:06.997Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1536gitlab # [ 279.807084] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:06.999Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8a235abc094e9041d31e74f7","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8a235abc094e9041d31e74f7: start","job_status":"start","queue_duration_s":1.215464,"scheduling_latency_s":1.215504}1537gitlab # [ 279.819940] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:07.036Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8a235abc094e9041d31e74f7","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8a235abc094e9041d31e74f7: done: 0.036854 sec","job_status":"done","queue_duration_s":1.215464,"scheduling_latency_s":1.215504,"redis_calls":13,"redis_duration_s":0.0030870000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001034,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002053,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026111,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.036854,"completed_at":"2026-11-13T10:38:07.036Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1538gitlab # [ 279.860317] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:07.039Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"456e6c8e83cf0322019a4593","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-456e6c8e83cf0322019a4593: start","job_status":"start","queue_duration_s":1.255344,"scheduling_latency_s":1.255398}1539gitlab # [ 279.873017] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:07.078Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"456e6c8e83cf0322019a4593","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-456e6c8e83cf0322019a4593: done: 0.038902 sec","job_status":"done","queue_duration_s":1.255344,"scheduling_latency_s":1.255398,"redis_calls":13,"redis_duration_s":0.0037689999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001478,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002291,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028558,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.038902,"completed_at":"2026-11-13T10:38:07.078Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1540gitlab # [ 279.913628] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:07.081Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"43171fcab14adda58b432ec4","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-43171fcab14adda58b432ec4: start","job_status":"start","queue_duration_s":1.296895,"scheduling_latency_s":1.296951}1541gitlab # [ 279.926716] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:07.115Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"43171fcab14adda58b432ec4","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-43171fcab14adda58b432ec4: done: 0.03434 sec","job_status":"done","queue_duration_s":1.296895,"scheduling_latency_s":1.296951,"redis_calls":13,"redis_duration_s":0.003394,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001075,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002319,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024399,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.03434,"completed_at":"2026-11-13T10:38:07.115Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1542gitlab # [ 279.967754] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:07.118Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"75fd422927651239965290ae","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-75fd422927651239965290ae: start","job_status":"start","queue_duration_s":1.33381,"scheduling_latency_s":1.333858}1543gitlab # [ 279.980721] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:07.150Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"75fd422927651239965290ae","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-75fd422927651239965290ae: done: 0.03193 sec","job_status":"done","queue_duration_s":1.33381,"scheduling_latency_s":1.333858,"redis_calls":13,"redis_duration_s":0.003032,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000965,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002067,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023261,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.03193,"completed_at":"2026-11-13T10:38:07.149Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1544gitlab # [ 280.020682] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:07.152Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6635b69aed8f77f09032827d","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6635b69aed8f77f09032827d: start","job_status":"start","queue_duration_s":1.367902,"scheduling_latency_s":1.367939}1545gitlab # [ 280.033376] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:07.188Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6635b69aed8f77f09032827d","created_at":"2026-11-13T10:38:05.700Z","trace_propagation_headers":{"sentry-trace":"c8d322d9c302428b9ddd73a020c32552-518473fc888f4999","baggage":"sentry-trace_id=c8d322d9c302428b9ddd73a020c32552,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ee39d7ef38b790588b8f479ec64935a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:05.784Z","job_size_bytes":2,"pid":1894,"sidekiq_tid":"mal6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6635b69aed8f77f09032827d: done: 0.036008 sec","job_status":"done","queue_duration_s":1.367902,"scheduling_latency_s":1.367939,"redis_calls":13,"redis_duration_s":0.003443,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001145,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002298,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027023,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.036008,"completed_at":"2026-11-13T10:38:07.188Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1546gitlab # [ 293.209134] postgres[3191]: [3191] ERROR: relation "table_without_models" does not exist1547gitlab # [ 293.209511] postgres[3191]: [3191] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1548gitlab # [ 298.766991] runuser[2850]: pam_unix(runuser:session): session closed for user gitlab1549gitlab # [ 298.768030] setup-gitlab-start[2819]: + systemctl restart gitlab-sidekiq.service1550gitlab # [ 298.791796] systemd[1]: Stopping gitlab-sidekiq.service...1551gitlab # [ 298.795225] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:27.742Z","message":"Shutting down"}1552gitlab # [ 298.797514] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:27.742Z","message":"Scheduler exiting..."}1553gitlab # [ 298.799697] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:27.743Z","message":"Terminating quiet threads for default capsule"}1554gitlab # [ 298.802544] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:27.743Z","message":"Scheduler exiting..."}1555gitlab # [ 299.295699] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:28.244Z","message":"Pausing to allow jobs to finish..."}1556gitlab # [ 303.799664] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:32.748Z","message":"Bye!"}1557gitlab # [ 303.827028] sidekiq-cluster[1894]: {"severity":"INFO","time":"2026-11-13T10:38:32.776Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":1894,"worker_id":"sidekiq_0","memwd_rss_bytes":915492864,"retry":0}1558gitlab # [ 304.650083] systemd[1]: gitlab-sidekiq.service: Deactivated successfully.1559gitlab # [ 304.650433] systemd[1]: Stopped gitlab-sidekiq.service.1560gitlab # [ 304.650550] systemd[1]: gitlab-sidekiq.service: Consumed 1min 47.139s CPU time over 3min 20.398s wall clock time, 941.6M memory peak, 5.3M read from disk.1561gitlab # [ 304.658124] systemd[1]: Started gitlab-sidekiq.service.1562gitlab # [ 304.661847] setup-gitlab-start[2819]: + mkdir -p /var/lib/secrets1563gitlab # [ 304.665015] setup-gitlab-start[2819]: + echo nixbot-test-token-of-20-chars1564gitlab # [ 304.666672] setup-gitlab-start[2819]: + chmod 644 /var/lib/secrets/gitlab-token1565gitlab # [ 304.668693] setup-gitlab-start[2819]: + timeout 300 bash -c $'until curl -fs -H "PRIVATE-TOKEN: $0" http://gitlab/api/v4/projects/root%2Ftest-flake > /dev/null; do\n curl -s -X POST -H "PRIVATE-TOKEN: $0" -H "Content-Type: application/json" \\\n -d "{\\"name\\":\\"test-flake\\",\\"visibility\\":\\"public\\",\\"topics\\":[\\"build-with-buildbot\\"]}" \\\n http://gitlab/api/v4/projects | jq -c "{id, message, error}"\n sleep 5\ndone' nixbot-test-token-of-20-chars1566gitlab # [ 305.085229] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:38:34 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 404 35 "" "curl/8.21.0" 3891567gitlab # [ 306.245488] gitaly[905]: time="2026-11-13T10:38:35.193Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7SP20RP4HW876FQ6BD330 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-13T10:38:45.001" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:38:35.193" pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1568gitlab # [ 306.265628] gitaly[905]: time="2026-11-13T10:38:35.194Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7SP20RP4HW876FQ6BD330 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-13T10:38:45.001" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:38:35.193" grpc.time_ms=0.93 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1569gitlab # [ 306.409512] gitaly[905]: time="2026-11-13T10:38:35.358Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7SP20RP4HW876FQ6BD330 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-13T10:38:45.167" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:38:35.358" pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1570gitlab # [ 306.426638] gitaly[905]: time="2026-11-13T10:38:35.358Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7SP20RP4HW876FQ6BD330 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-13T10:38:45.167" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:38:35.358" grpc.time_ms=0.39 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1571gitlab # [ 306.473901] gitaly[905]: time="2026-11-13T10:38:35.423Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7SP20RP4HW876FQ6BD330 diskcache=941f7c68-376a-4809-a612-f83ec43782e1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-11-13T10:38:45.192" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:38:35.383" pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1572gitlab # [ 306.490613] gitaly[905]: time="2026-11-13T10:38:35.423Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=19 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1440 command.oublock=32 command.real_time_ms=23 command.spawn_token_fork_ms=1 command.system_time_ms=13 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7SP20RP4HW876FQ6BD330 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-11-13T10:38:45.192" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:38:35.383" grpc.time_ms=39.891 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1573gitlab # [ 306.509591] gitaly[905]: time="2026-11-13T10:38:35.437Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7SP20RP4HW876FQ6BD330 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-13T10:38:45.246" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:38:35.437" pid=905 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1574gitlab # [ 306.522312] gitaly[905]: time="2026-11-13T10:38:35.438Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7SP20RP4HW876FQ6BD330 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-13T10:38:45.246" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:38:35.437" grpc.time_ms=0.9 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1575gitlab # [ 306.535562] gitaly[905]: time="2026-11-13T10:38:35.452Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7SP20RP4HW876FQ6BD330 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-13T10:38:45.252" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:38:35.443" grpc.time_ms=9.463 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1576gitlab # [ 306.591024] gitaly[905]: time="2026-11-13T10:38:35.540Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7SP20RP4HW876FQ6BD330 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-13T10:38:45.009" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:38:35.540" pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1577gitlab # [ 306.603921] gitaly[905]: time="2026-11-13T10:38:35.540Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7SP20RP4HW876FQ6BD330 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-13T10:38:45.009" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:38:35.540" grpc.time_ms=0.34 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1578gitlab # [ 306.627768] gitaly[905]: time="2026-11-13T10:38:35.577Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7SP20RP4HW876FQ6BD330 diskcache=dab1ccae-0fc4-420f-8751-7b633d1236dd grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-11-13T10:38:45.013" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:38:35.543" pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1579gitlab # [ 306.644129] gitaly[905]: time="2026-11-13T10:38:35.577Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1439 command.oublock=32 command.real_time_ms=20 command.spawn_token_fork_ms=0 command.system_time_ms=12 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7SP20RP4HW876FQ6BD330 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-11-13T10:38:45.013" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:38:35.543" grpc.time_ms=33.579 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1580gitlab # [ 306.661918] gitaly[905]: time="2026-11-13T10:38:35.585Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7SP20RP4HW876FQ6BD330 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-13T10:38:45.054" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:38:35.584" pid=905 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1581gitlab # [ 306.675177] gitaly[905]: time="2026-11-13T10:38:35.585Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7SP20RP4HW876FQ6BD330 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-13T10:38:45.054" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:38:35.584" grpc.time_ms=0.569 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1582gitlab # [ 306.687750] gitaly[905]: time="2026-11-13T10:38:35.598Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7SP20RP4HW876FQ6BD330 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-13T10:38:45.058" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:38:35.589" grpc.time_ms=9.207 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1583gitlab # [ 306.703241] gitaly[905]: time="2026-11-13T10:38:35.636Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7SP20RP4HW876FQ6BD330 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-13T10:38:45.104" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:38:35.635" grpc.time_ms=0.215 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1584gitlab # [ 306.714909] gitaly[905]: time="2026-11-13T10:38:35.645Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7SP20RP4HW876FQ6BD330 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-11-13T10:38:45.114" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:38:35.645" grpc.time_ms=0.376 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1585gitlab # [ 307.255981] gitaly[905]: time="2026-11-13T10:38:36.204Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7SP20RP4HW876FQ6BD330 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-13T10:38:46.003" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:38:36.194" grpc.time_ms=9.807 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1586gitlab # [ 307.635129] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:38:36 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 25341587gitlab # [ 307.638954] setup-gitlab-start[3269]: {"id":1,"message":null,"error":null}1588gitlab # [ 308.094275] redis[784]: 10 changes in 300 seconds. Saving...1589gitlab # [ 308.095987] redis[784]: Background saving started by pid 33151590gitlab # [ 308.100425] redis[3315]: BGSAVE done, 392 keys saved, 0 keys skipped, 103817 bytes written.1591gitlab # [ 308.104777] redis[3315]: DB saved on disk1592gitlab # [ 308.105971] redis[3315]: Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB1593gitlab # [ 308.194976] redis[784]: Background saving terminated with success1594gitlab # [ 309.724181] sidekiq-cluster[3253]: {"severity":"INFO","time":"2026-11-13T10:38:38.644Z","message":"Starting cluster with 1 processes"}1595gitlab # [ 309.929921] postgres[859]: [859] LOG: checkpoint starting: time1596gitlab # [ 310.457513] gitaly[905]: time="2026-11-13T10:38:39.406Z" level=info msg="burden monitor stats" pid=9051597gitlab # [ 312.822464] gitaly[905]: time="2026-11-13T10:38:41.770Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=5 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7SXEERSXGTVMEBP9X32Q3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-13T10:38:51.226" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:38:41.756" grpc.time_ms=13.468 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1598gitlab # [ 312.927073] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:38:41 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 200 4924 "" "curl/8.21.0" 2611599gitlab # [ 312.931106] setup-gitlab-start[2819]: + mkdir -p /tmp/test-flake1600gitlab # [ 312.934157] setup-gitlab-start[2819]: + cd /tmp/test-flake1601gitlab # [ 312.935057] setup-gitlab-start[2819]: + git init -b master1602gitlab # [ 312.954203] setup-gitlab-start[3344]: Initialized empty Git repository in /tmp/test-flake/.git/1603gitlab # [ 312.955498] setup-gitlab-start[2819]: + git config user.name test1604gitlab # [ 312.958928] setup-gitlab-start[2819]: + git config user.email test@example.com1605gitlab # [ 312.963492] setup-gitlab-start[2819]: + cat1606gitlab # [ 312.967148] setup-gitlab-start[2819]: + git add flake.nix1607gitlab # [ 312.972603] setup-gitlab-start[2819]: + git commit -m 'initial commit'1608gitlab # [ 312.994446] setup-gitlab-start[3349]: [master (root-commit) 5c12ba2] initial commit1609gitlab # [ 312.996845] setup-gitlab-start[3349]: 1 file changed, 10 insertions(+)1610gitlab # [ 312.998287] setup-gitlab-start[3349]: create mode 100644 flake.nix1611gitlab # [ 312.999666] setup-gitlab-start[2819]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git1612gitlab # [ 313.004383] setup-gitlab-start[2819]: + git push -u origin master1613gitlab # [ 313.115277] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:38:42 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 791614gitlab # [ 313.269075] gitaly[905]: time="2026-11-13T10:38:42.218Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=581 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7SXWH4AMD4N3GARVVJM75 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:38:42.207" grpc.time_ms=10.434 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1615gitlab # [ 313.284383] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:38:42 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 1531616gitlab # [ 313.863184] gitaly[905]: time="2026-11-13T10:38:42.811Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=3 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7SY1QCQ856G1GYR454CPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-13T10:38:52.002" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:38:42.802" grpc.time_ms=8.913 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1617gitlab # [ 313.910052] gitaly[905]: time="2026-11-13T10:38:42.859Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=301 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7SY1QCQ856G1GYR454CPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-11-13T10:39:12.048" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-11-13T10:38:42.848" grpc.time_ms=10.374 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1618gitlab # [ 314.000532] gitaly[905]: time="2026-11-13T10:38:42.949Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M7A7SY1QCQ856G1GYR454CPX duration_ms=580ns method=POST pid=905 status=200 url="http://unix/api/v4/internal/allowed"1619gitlab # [ 314.041804] gitaly[905]: time="2026-11-13T10:38:42.991Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M7A7SY1QCQ856G1GYR454CPX duration_ms=40ns method=POST pid=905 status=200 url="http://unix/api/v4/internal/pre_receive"1620gitlab # [ 314.051111] gitaly[905]: time="2026-11-13T10:38:42.991Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7SY1QCQ856G1GYR454CPX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5315 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:38:42.367" grpc.time_ms=623.651 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1621gitlab # [ 314.077516] gitaly[905]: time="2026-11-13T10:38:43.026Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7SY1QCQ856G1GYR454CPX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4785 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:38:43.025" grpc.time_ms=1.362 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1622gitlab # [ 314.108783] gitaly[905]: time="2026-11-13T10:38:43.057Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7SY1QCQ856G1GYR454CPX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4784 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:38:43.057" grpc.time_ms=0.763 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1623gitlab # [ 314.150681] gitaly[905]: time="2026-11-13T10:38:43.099Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7SY1QCQ856G1GYR454CPX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4786 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:38:43.099" grpc.time_ms=0.577 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1624gitlab # [ 314.332718] gitaly[905]: time="2026-11-13T10:38:43.281Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=576 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7SY1QCQ856G1GYR454CPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-13T10:38:53.010" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:38:43.271" grpc.time_ms=10.573 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1625gitlab # [ 314.363447] gitaly[905]: time="2026-11-13T10:38:43.312Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1433 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=4 command.system_time_ms=3 command.user_time_ms=10 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7SY1QCQ856G1GYR454CPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-11-13T10:38:53.031" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:38:43.292" grpc.time_ms=20.276 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1626gitlab # [ 314.386100] gitaly[905]: time="2026-11-13T10:38:43.324Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M7A7SY1QCQ856G1GYR454CPX duration_ms=177ns method=POST pid=905 status=200 url="http://unix/api/v4/internal/post_receive"1627gitlab # [ 314.390942] gitaly[905]: time="2026-11-13T10:38:43.325Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7SY1QCQ856G1GYR454CPX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4809 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:38:43.144" grpc.time_ms=181.023 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1628gitlab # [ 314.407366] gitaly[905]: time="2026-11-13T10:38:43.334Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M7A7SY1QCQ856G1GYR454CPX diskcache=04687b97-47d6-4d2c-84c4-28871c552715 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:38:42.299" pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1629gitlab # [ 314.421931] gitlab-workhorse[818]: time="2026-11-13T10:38:43Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521630gitlab # [ 314.424285] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:38:43 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 11031631gitlab # [ 314.427844] systemd[1]: Finished setup-gitlab.service.1632gitlab # [ 314.429038] setup-gitlab-start[3353]: To http://gitlab/root/test-flake.git1633gitlab # [ 314.430173] setup-gitlab-start[3353]: * [new branch] master -> master1634gitlab # [ 314.431321] setup-gitlab-start[3353]: branch 'master' set up to track 'origin/master'.1635gitlab # [ 314.432686] gitaly[905]: time="2026-11-13T10:38:43.334Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=197 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=851968 command.minflt=13449 command.oublock=32 command.real_time_ms=1027 command.spawn_token_fork_ms=0 command.system_time_ms=101 command.user_time_ms=96 component=gitaly.StreamServerInterceptor correlation_id=01M7A7SY1QCQ856G1GYR454CPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:38:42.299" grpc.time_ms=1034.462 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1636gitlab # [ 314.449628] systemd[1]: Starting nixbot CI service...1637gitlab: (finished: waiting for unit setup-gitlab.service, in 315.83 seconds)1638gitlab: waiting for unit nixbot.service1639gitlab # [ 315.482305] nixbot[3443]: {"ts": "2026-11-13T10:38:44.430Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1640gitlab # [ 315.509705] nixbot[3443]: {"ts": "2026-11-13T10:38:44.458Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1641gitlab # [ 315.532499] nixbot[3443]: {"ts": "2026-11-13T10:38:44.481Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1642gitlab # [ 315.536732] nixbot[3443]: {"ts": "2026-11-13T10:38:44.485Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1643gitlab # [ 315.540302] nixbot[3443]: {"ts": "2026-11-13T10:38:44.489Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1644gitlab # [ 315.545502] nixbot[3443]: {"ts": "2026-11-13T10:38:44.494Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1645gitlab # [ 315.549413] nixbot[3443]: {"ts": "2026-11-13T10:38:44.498Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1646gitlab # [ 315.552852] nixbot[3443]: {"ts": "2026-11-13T10:38:44.500Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1647gitlab # [ 315.556267] nixbot[3443]: {"ts": "2026-11-13T10:38:44.505Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1648gitlab # [ 315.559785] nixbot[3443]: {"ts": "2026-11-13T10:38:44.506Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1649gitlab # [ 315.563470] nixbot[3443]: {"ts": "2026-11-13T10:38:44.510Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1650gitlab # [ 315.567048] nixbot[3443]: {"ts": "2026-11-13T10:38:44.515Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1651gitlab # [ 315.570738] nixbot[3443]: {"ts": "2026-11-13T10:38:44.518Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1652gitlab # [ 315.574363] nixbot[3443]: {"ts": "2026-11-13T10:38:44.520Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1653gitlab # [ 315.578109] nixbot[3443]: {"ts": "2026-11-13T10:38:44.523Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1654gitlab # [ 315.581665] nixbot[3443]: {"ts": "2026-11-13T10:38:44.525Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1655gitlab # [ 315.585123] nixbot[3443]: {"ts": "2026-11-13T10:38:44.526Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1656gitlab # [ 315.588651] nixbot[3443]: {"ts": "2026-11-13T10:38:44.527Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1657gitlab # [ 315.592119] nixbot[3443]: {"ts": "2026-11-13T10:38:44.531Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"}1658gitlab # [ 315.595561] nixbot[3443]: {"ts": "2026-11-13T10:38:44.532Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"}1659gitlab # [ 315.599127] nixbot[3443]: {"ts": "2026-11-13T10:38:44.533Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"}1660gitlab # [ 315.602336] nixbot[3443]: {"ts": "2026-11-13T10:38:44.537Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"}1661gitlab # [ 315.605535] nixbot[3443]: {"ts": "2026-11-13T10:38:44.538Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"}1662gitlab # [ 315.608620] nixbot[3443]: {"ts": "2026-11-13T10:38:44.540Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"}1663gitlab # [ 315.611700] nixbot[3443]: {"ts": "2026-11-13T10:38:44.540Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"}1664gitlab # [ 315.615093] nixbot[3443]: {"ts": "2026-11-13T10:38:44.542Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"}1665gitlab # [ 315.618346] nixbot[3443]: {"ts": "2026-11-13T10:38:44.543Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"}1666gitlab # [ 315.621413] nixbot[3443]: {"ts": "2026-11-13T10:38:44.544Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"}1667gitlab # [ 315.624229] nixbot[3443]: {"ts": "2026-11-13T10:38:44.545Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"}1668gitlab # [ 315.627055] nixbot[3443]: {"ts": "2026-11-13T10:38:44.555Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"}1669gitlab # [ 315.629987] nixbot[3443]: {"ts": "2026-11-13T10:38:44.558Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"}1670gitlab # [ 315.632824] nixbot[3443]: {"ts": "2026-11-13T10:38:44.560Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"}1671gitlab # [ 315.635548] nixbot[3443]: {"ts": "2026-11-13T10:38:44.561Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 32, "migration": "upload_queue"}1672gitlab # [ 315.638338] nixbot[3443]: {"ts": "2026-11-13T10:38:44.564Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 33, "migration": "effect_runs_catchup"}1673gitlab # [ 315.641066] nixbot[3443]: {"ts": "2026-11-13T10:38:44.565Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 34, "migration": "public_badges"}1674gitlab # [ 315.643620] nixbot[3443]: {"ts": "2026-11-13T10:38:44.571Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 35, "migration": "settle_reset_effects"}1675gitlab # [ 315.646396] nixbot[3443]: {"ts": "2026-11-13T10:38:44.574Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 36, "migration": "pr_approvals"}1676gitlab # [ 315.649744] nixbot[3443]: {"ts": "2026-11-13T10:38:44.578Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 37, "migration": "effect_when_changed"}1677gitlab # [ 315.652221] nixbot[3443]: {"ts": "2026-11-13T10:38:44.581Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 38, "migration": "commit_statuses"}1678gitlab # [ 316.041936] nixbot[3443]: {"ts": "2026-11-13T10:38:44.991Z", "level": "info", "logger": "uvicorn.error", "msg": "Started server process [3443]", "color_message": "Started server process [\u001b[36m%d\u001b[0m]"}1679gitlab # [ 316.046074] nixbot[3443]: {"ts": "2026-11-13T10:38:44.991Z", "level": "info", "logger": "uvicorn.error", "msg": "Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)"}1680gitlab # [ 316.066964] nixbot[3443]: {"ts": "2026-11-13T10:38:45.015Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:41790 - \"GET /health HTTP/1.1\" 200"}1681gitlab # [ 316.073659] systemd[1]: Started nixbot CI service.1682gitlab # [ 316.074988] systemd[1]: Reached target Multi-User System.1683gitlab # [ 316.077782] systemd[1]: Startup finished in 661ms (kernel) + 3.621s (initrd) + 5min 11.793s (userspace) = 5min 16.076s.1684gitlab: (finished: waiting for unit nixbot.service, in 1.05 seconds)1685gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1686gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.03 seconds)1687subtest: project discovered and webhook registered1688gitlab # [ 316.487472] nixbot[3443]: {"ts": "2026-11-13T10:38:45.435Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:41794 - \"GET /health HTTP/1.1\" 200"}1689gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1690gitlab # [ 316.597350] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:38:45 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.21.0" 781691gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.11 seconds)1692gitlab # [ 317.224124] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:38:46 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 11551693gitlab # [ 317.247869] nixbot[3443]: {"ts": "2026-11-13T10:38:46.196Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1694gitlab # [ 317.344418] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:38:46 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 861695gitlab # [ 317.347858] nixbot[3443]: {"ts": "2026-11-13T10:38:46.296Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1696gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1697gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.11 seconds)1698gitlab # [ 317.711684] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:38:46 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.21.0" 811699gitlab # [ 317.783765] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:38:46 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 917 "" "python-httpx/0.28.1" 4321700gitlab # [ 317.971947] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:38:46 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 1781701gitlab # [ 318.035994] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:38:46 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 880 "" "python-httpx/0.28.1" 551702gitlab # [ 318.135521] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:38:47 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 917 "" "python-httpx/0.28.1" 931703gitlab # [ 318.232559] gitaly[905]: time="2026-11-13T10:38:47.181Z" level=info msg="finished unary call with code OK" catfile.duration_ms=3 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=3 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7T2SMYMKKXE4TNMNRGJ5H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-11-13T10:39:17.063" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:38:47.164" grpc.time_ms=16.185 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1704gitlab # [ 318.255986] gitaly[905]: time="2026-11-13T10:38:47.199Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7T2SMYMKKXE4TNMNRGJ5H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:39:17.098" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:38:47.198" grpc.time_ms=0.891 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1705gitlab # [ 318.272240] gitaly[905]: time="2026-11-13T10:38:47.216Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=3 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7T2SMYMKKXE4TNMNRGJ5H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-11-13T10:38:57.014" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:38:47.205" grpc.time_ms=10.701 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1706gitlab # [ 318.310893] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:38:47 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 1671707gitlab # [ 318.411676] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:38:47 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 951708gitlab # [ 318.417738] nixbot[3443]: {"ts": "2026-11-13T10:38:47.366Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1709gitlab # [ 318.562103] gitaly[905]: time="2026-11-13T10:38:47.511Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M7A7T34FVJ7X7R6JP77AC35W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:38:47.500" grpc.time_ms=10.612 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1710gitlab # [ 318.579124] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:38:47 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 731711gitlab # [ 318.682783] gitaly[905]: time="2026-11-13T10:38:47.632Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7T36TTK6VXXD9XGTQMPD9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:38:47.621" pid=905 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc1712gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1713gitlab # [ 318.699889] gitaly[905]: time="2026-11-13T10:38:47.632Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7T36TTK6VXXD9XGTQMPD9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:38:47.621" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1714gitlab # [ 318.723354] gitlab-workhorse[818]: time="2026-11-13T10:38:47Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1131715gitlab # [ 318.726595] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:38:47 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 1181716gitlab # [ 318.729999] gitaly[905]: time="2026-11-13T10:38:47.632Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7T36TTK6VXXD9XGTQMPD9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:38:47.621" grpc.time_ms=11.146 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1717gitlab # [ 318.770130] gitaly[905]: time="2026-11-13T10:38:47.719Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7T3ANFSE9GCG6W5CQ51T0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:38:47.700" pid=905 remote_ip="2001:db8:1::1" request_sha=76ef90d445a5d51e44f5923a9c9dcee4c5642c53 response_bytes=224 span.kind=server system=grpc1718gitlab # [ 318.783029] gitaly[905]: time="2026-11-13T10:38:47.719Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1139 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=11 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7T3ANFSE9GCG6W5CQ51T0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:38:47.700" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1719gitlab # [ 318.801036] gitlab-workhorse[818]: time="2026-11-13T10:38:47Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2241720gitlab # [ 318.803822] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:38:47 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 224 "" "git/2.55.0" 831721gitlab # [ 318.806414] gitaly[905]: time="2026-11-13T10:38:47.719Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1139 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=11 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7T3ANFSE9GCG6W5CQ51T0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:38:47.700" grpc.time_ms=18.997 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1722gitlab # [ 318.824244] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:38:47 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 880 "" "curl/8.21.0" 741723gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.12 seconds)1724(finished: subtest: project discovered and webhook registered, in 2.34 seconds)1725subtest: push delivers the webhook1726gitlab: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails'1727gitlab # [ 318.886795] gitaly[905]: time="2026-11-13T10:38:47.835Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M7A7T3EQWJMD1M2DNMWQV4M8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:38:47.826" grpc.time_ms=9.289 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1728gitlab # [ 318.901010] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:38:47 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 681729gitlab # [ 318.970838] gitaly[905]: time="2026-11-13T10:38:47.920Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7T3GXX7HEE84W49KNDFB4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:38:47.909" pid=905 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1730gitlab # [ 318.982030] gitaly[905]: time="2026-11-13T10:38:47.930Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7T3GXX7HEE84W49KNDFB4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:38:47.909" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1731gitlab # [ 318.996038] gitaly[905]: time="2026-11-13T10:38:47.931Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7T3GXX7HEE84W49KNDFB4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:38:47.909" grpc.time_ms=21.114 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1732gitlab # [ 319.011075] gitlab-workhorse[818]: time="2026-11-13T10:38:47Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671733gitlab # [ 319.013042] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:38:47 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 931734gitlab # [ 319.100221] gitaly[905]: time="2026-11-13T10:38:48.049Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7T3N54S77GKF10RQDAK4V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:38:48.036" grpc.time_ms=12.921 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1735gitlab # [ 319.122780] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:38:48 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 761736gitlab # [ 319.184044] gitaly[905]: time="2026-11-13T10:38:48.132Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7T3QM875XGY5J0E6YBA6F grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:38:48.113" pid=905 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc1737gitlab # [ 319.201125] gitaly[905]: time="2026-11-13T10:38:48.132Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1100 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7T3QM875XGY5J0E6YBA6F grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:38:48.113" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1738gitlab # [ 319.223366] gitlab-workhorse[818]: time="2026-11-13T10:38:48Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2091739gitlab # [ 319.226481] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:38:48 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 811740gitlab # [ 319.229618] gitaly[905]: time="2026-11-13T10:38:48.133Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1100 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7T3QM875XGY5J0E6YBA6F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:38:48.113" grpc.time_ms=19.373 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1741gitlab # [ 319.294340] gitaly[905]: time="2026-11-13T10:38:48.242Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7T3VBGCY888589SVN5M8J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:39:18.141" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:38:48.242" grpc.time_ms=0.759 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1742gitlab # [ 319.334202] gitaly[905]: time="2026-11-13T10:38:48.283Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=588 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7T3VBGCY888589SVN5M8J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-11-13T10:39:18.172" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:38:48.272" grpc.time_ms=10.966 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1743gitlab # [ 319.360823] gitaly[905]: time="2026-11-13T10:38:48.309Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M7A7T3VBGCY888589SVN5M8J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-13T10:38:58.109" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:38:48.300" grpc.time_ms=9.285 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1744gitlab # [ 319.381691] gitaly[905]: time="2026-11-13T10:38:48.330Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7T3VBGCY888589SVN5M8J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-13T10:38:58.130" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:38:48.321" grpc.time_ms=9.416 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1745gitlab # [ 319.469187] gitaly[905]: time="2026-11-13T10:38:48.418Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7T3VBGCY888589SVN5M8J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-11-13T10:38:58.227" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:38:48.417" grpc.time_ms=0.429 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1746gitlab # [ 319.488046] gitaly[905]: time="2026-11-13T10:38:48.437Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=570 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7T3VBGCY888589SVN5M8J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-11-13T10:38:58.236" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:38:48.427" grpc.time_ms=9.381 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1747gitlab # [ 319.534993] gitaly[905]: time="2026-11-13T10:38:48.484Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7T3VBGCY888589SVN5M8J grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4153 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:38:48.482" grpc.time_ms=1.324 pid=905 span.kind=server system=grpc1748gitlab # [ 319.568063] gitaly[905]: time="2026-11-13T10:38:48.517Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7T3VBGCY888589SVN5M8J grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:38:48.516" grpc.time_ms=0.47 pid=905 span.kind=server system=grpc1749gitlab # [ 319.583949] gitaly[905]: time="2026-11-13T10:38:48.526Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7T3VBGCY888589SVN5M8J diskcache=b220dbec-109c-4cae-9bcf-43401eb7b25c grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-13T10:38:58.249" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:38:48.440" pid=905 ref=refs/keep-around/5c12ba2ba5cde5b1dd1bfa095c2839632222f25c remote_ip="2001:db8:1::1" revision=5c12ba2ba5cde5b1dd1bfa095c2839632222f25c span.kind=server system=grpc username=root1750gitlab # [ 319.602508] gitaly[905]: time="2026-11-13T10:38:48.526Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=79 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5454 command.oublock=8 command.real_time_ms=81 command.spawn_token_fork_ms=0 command.system_time_ms=38 command.user_time_ms=41 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7T3VBGCY888589SVN5M8J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-13T10:38:58.249" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:38:48.440" grpc.time_ms=86.479 pid=905 ref=refs/keep-around/5c12ba2ba5cde5b1dd1bfa095c2839632222f25c remote_ip="2001:db8:1::1" revision=5c12ba2ba5cde5b1dd1bfa095c2839632222f25c span.kind=server system=grpc username=root1751gitlab # [ 319.622961] gitaly[905]: time="2026-11-13T10:38:48.531Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7T3VBGCY888589SVN5M8J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:39:18.430" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:38:48.530" grpc.time_ms=0.348 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1752gitlab # [ 319.765694] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:38:48 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/5c12ba2ba5cde5b1dd1bfa095c2839632222f25c HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 5421753gitlab # [ 319.779117] nixbot[3443]: {"ts": "2026-11-13T10:38:48.727Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-5c12ba2ba5cd-0"}1754gitlab # [ 319.978031] nixbot[3443]: {"ts": "2026-11-13T10:38:48.927Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-5c12ba2ba5cd-0", "argv0": "bwrap"}1755gitlab # [ 320.109454] systemd[1]: Started Nix Daemon.1756gitlab # [ 320.191917] nix-daemon[3607]: accepted connection from pid 3602, user nixbot1757gitlab # [ 325.296800] nix-daemon[3607]: accepted connection from pid 3657, user nixbot1758gitlab # [ 325.315119] nix-daemon[3607]: accepted connection from pid 3602, user nixbot1759gitlab # [ 330.026381] nix-daemon[3607]: accepted connection from pid 3602, user nixbot1760gitlab # [ 330.163147] gitaly[905]: time="2026-11-13T10:38:59.111Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7TEERPARVMCCY7R4YY1PR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:39:29.010" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:38:59.110" grpc.time_ms=0.841 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1761gitlab # [ 330.313876] gitaly[905]: time="2026-11-13T10:38:59.263Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M7A7TEERPARVMCCY7R4YY1PR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-13T10:39:09.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:38:59.252" grpc.time_ms=10.134 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1762gitlab # [ 330.441017] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:38:59 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/5c12ba2ba5cde5b1dd1bfa095c2839632222f25c HTTP/1.1" 201 676 "" "python-httpx/0.28.1" 3561763gitlab # [ 330.516852] gitaly[905]: time="2026-11-13T10:38:59.466Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7TET62M5R499P8V4WMP61 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:39:29.364" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:38:59.465" grpc.time_ms=0.791 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1764gitlab # [ 330.557019] gitaly[905]: time="2026-11-13T10:38:59.506Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7TET62M5R499P8V4WMP61 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-13T10:39:09.235" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:38:59.496" grpc.time_ms=9.825 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1765gitlab # [ 330.616729] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:38:59 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/5c12ba2ba5cde5b1dd1bfa095c2839632222f25c HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1661766gitlab # [ 330.688132] gitaly[905]: time="2026-11-13T10:38:59.637Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7TEZWGWEQW4XKEVJWYE7F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:39:29.535" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:38:59.636" grpc.time_ms=0.76 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1767gitlab # [ 330.730546] gitaly[905]: time="2026-11-13T10:38:59.679Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M7A7TEZWGWEQW4XKEVJWYE7F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-13T10:39:09.010" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:38:59.670" grpc.time_ms=9.224 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1768gitlab # [ 330.792873] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:38:59 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/5c12ba2ba5cde5b1dd1bfa095c2839632222f25c HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1601769gitlab # [ 331.054183] nixbot[3443]: {"ts": "2026-11-13T10:39:00.002Z", "level": "info", "logger": "nixbot.effects_run", "msg": "effects discovered", "build_id": 1, "sha": "5c12ba2ba5cde5b1dd1bfa095c2839632222f25c", "branch": "master", "pr": null, "allowed": true, "effects_started": false, "effects": []}1770gitlab # [ 331.168589] gitaly[905]: time="2026-11-13T10:39:00.116Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=8 component=gitaly.StreamServerInterceptor correlation_id=01M7A7TFE834HJNHB2VRNGSFY8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:39:00.105" grpc.time_ms=10.915 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1771gitlab # [ 331.186251] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:39:00 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 751772gitlab # [ 331.254275] gitaly[905]: time="2026-11-13T10:39:00.203Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7TFGPRAP2SX95XT98C6DJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:39:00.190" pid=905 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1773gitlab # [ 331.272368] gitaly[905]: time="2026-11-13T10:39:00.203Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=586 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7TFGPRAP2SX95XT98C6DJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:39:00.190" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1774gitlab # [ 331.293521] gitlab-workhorse[818]: time="2026-11-13T10:39:00Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671775gitlab # [ 331.296218] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:39:00 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 851776gitlab # [ 331.298788] gitaly[905]: time="2026-11-13T10:39:00.204Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=586 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7TFGPRAP2SX95XT98C6DJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:39:00.190" grpc.time_ms=13.758 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1777gitlab # [ 331.399827] gitaly[905]: time="2026-11-13T10:39:00.348Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=577 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7TFNAHPV24M69XGQTM8DE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:39:00.339" grpc.time_ms=8.468 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1778gitlab # [ 331.422486] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:39:00 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 821779gitlab # [ 331.500081] gitaly[905]: time="2026-11-13T10:39:00.449Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7TFRJE6R89J0P0M9VDZDB grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:39:00.438" pid=905 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc1780gitlab # [ 331.516232] gitaly[905]: time="2026-11-13T10:39:00.449Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7TFRJE6R89J0P0M9VDZDB grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:39:00.438" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1781gitlab # [ 331.534674] gitlab-workhorse[818]: time="2026-11-13T10:39:00Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671782gitlab # [ 331.537003] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:39:00 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 791783gitlab # [ 331.539263] gitaly[905]: time="2026-11-13T10:39:00.449Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7TFRJE6R89J0P0M9VDZDB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:39:00.438" grpc.time_ms=11.118 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1784gitlab # [ 351.072295] sidekiq-cluster[3321]: 2026-11-13T10:39:20.020Z pid=3321 tid=yfl INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>3321, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>432422912}1785gitlab # [ 351.359395] sidekiq-cluster[3321]: W, [2026-11-13T10:39:20.308485 #3321] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1786gitlab # [ 351.424375] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:39:20.373Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1787gitlab # [ 354.933617] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:39:23.882Z","message":"GitLab reliable fetch activated!"}1788gitlab # [ 354.933889] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:39:23.882Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1789gitlab # [ 391.153078] postgres[4211]: [4211] ERROR: relation "table_without_models" does not exist1790gitlab # [ 391.153545] postgres[4211]: [4211] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1791gitlab # [ 396.011862] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:04.960Z","message":"Booted Rails 7.2.3.1 application in production environment"}1792gitlab # [ 396.016749] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:04.965Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1793gitlab # [ 396.018877] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:04.966Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1794gitlab # [ 396.020755] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:04.966Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1795gitlab # [ 396.022886] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:04.968Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1796gitlab: (finished: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails', in 77.62 seconds)1797gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1798gitlab # [ 396.579050] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:40:05 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 761799gitlab # [ 396.673550] gitaly[905]: time="2026-11-13T10:40:05.622Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7WFCR9XZQWSGJF3MXEHRF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:40:05.611" grpc.time_ms=10.493 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1800gitlab # Enumerating objects: 5, done.1801gitlab # [ 396.695396] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:40:05 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 202 "" "git/2.55.0" 931802gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1803gitlab # Delta compression using up to 4 threads1804gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1805gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 274 bytes | 137.00 KiB/s, done.1806gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1807gitlab # [ 396.962139] gitaly[905]: time="2026-11-13T10:40:05.911Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=300 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7WFGXP3SWPPDZ22MN1RGB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-11-13T10:40:35.800" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-11-13T10:40:05.901" grpc.time_ms=10.105 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1808gitlab # [ 396.986495] gitaly[905]: time="2026-11-13T10:40:05.935Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M7A7WFGXP3SWPPDZ22MN1RGB duration_ms=131ns method=POST pid=905 status=200 url="http://unix/api/v4/internal/allowed"1809gitlab # [ 397.014055] gitaly[905]: time="2026-11-13T10:40:05.963Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M7A7WFGXP3SWPPDZ22MN1RGB duration_ms=26ns method=POST pid=905 status=200 url="http://unix/api/v4/internal/pre_receive"1810gitlab # [ 397.021314] gitaly[905]: time="2026-11-13T10:40:05.963Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7WFGXP3SWPPDZ22MN1RGB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5315 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:40:05.803" grpc.time_ms=159.822 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1811gitlab # [ 397.047869] gitaly[905]: time="2026-11-13T10:40:05.997Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7WFGXP3SWPPDZ22MN1RGB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4785 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:40:05.996" grpc.time_ms=0.729 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1812gitlab # [ 397.095419] gitaly[905]: time="2026-11-13T10:40:06.044Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7WFGXP3SWPPDZ22MN1RGB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4784 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:40:06.043" grpc.time_ms=0.772 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1813gitlab # [ 397.127369] gitaly[905]: time="2026-11-13T10:40:06.076Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7WFGXP3SWPPDZ22MN1RGB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4786 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:40:06.075" grpc.time_ms=0.632 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1814gitlab # [ 397.302002] gitaly[905]: time="2026-11-13T10:40:06.251Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M7A7WFGXP3SWPPDZ22MN1RGB duration_ms=143ns method=POST pid=905 status=200 url="http://unix/api/v4/internal/post_receive"1815gitlab # [ 397.308464] gitaly[905]: time="2026-11-13T10:40:06.252Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7WFGXP3SWPPDZ22MN1RGB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4809 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:40:06.107" grpc.time_ms=144.694 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1816gitlab # To http://gitlab/root/test-flake.git1817gitlab # [ 397.321942] gitaly[905]: time="2026-11-13T10:40:06.264Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M7A7WFGXP3SWPPDZ22MN1RGB diskcache=f7fcdd98-16fe-4cd0-8cdd-9e50b19d7416 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:40:05.744" pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1818gitlab # 5c12ba2..6f5cdc2 [ 397.334261] gitlab-workhorse[818]: time="2026-11-13T10:40:06Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521819gitlab # master -> master[ 397.336627] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:40:06 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 6031820gitlab # 1821gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 0.90 seconds)1822gitlab: must succeed: git -C /tmp/test-flake rev-parse master1823gitlab # [ 397.338895] gitaly[905]: time="2026-11-13T10:40:06.264Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=200 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=847872 command.minflt=13302 command.oublock=32 command.real_time_ms=514 command.spawn_token_fork_ms=0 command.system_time_ms=100 command.user_time_ms=100 component=gitaly.StreamServerInterceptor correlation_id=01M7A7WFGXP3SWPPDZ22MN1RGB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=606 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=58 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:40:05.744" grpc.time_ms=520.044 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1824gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.02 seconds)1825gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5'1826gitlab # [ 397.514913] nixbot[3443]: {"ts": "2026-11-13T10:40:06.463Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:54934 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5 HTTP/1.1\" 200"}1827gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5', in 0.16 seconds)1828gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5'1829gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5', in 0.03 seconds)1830gitlab # [ 398.549629] nixbot[3443]: {"ts": "2026-11-13T10:40:07.498Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:54950 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5 HTTP/1.1\" 200"}1831gitlab # [ 398.689687] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:07.637Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"19a15b52e167098b4804bf44","created_at":"2026-11-13T10:38:35.465Z","trace_propagation_headers":{"sentry-trace":"d6df8a21b51d434499cababb3aba092d-d920e4f484b5402e","baggage":"sentry-trace_id=d6df8a21b51d434499cababb3aba092d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7SP20RP4HW876FQ6BD330","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:35.511Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-19a15b52e167098b4804bf44: start","job_status":"start","queue_duration_s":92.126183,"scheduling_latency_s":92.126246}1832gitlab # [ 398.911423] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:07.859Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"19a15b52e167098b4804bf44","created_at":"2026-11-13T10:38:35.465Z","trace_propagation_headers":{"sentry-trace":"d6df8a21b51d434499cababb3aba092d-d920e4f484b5402e","baggage":"sentry-trace_id=d6df8a21b51d434499cababb3aba092d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7SP20RP4HW876FQ6BD330","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:35.511Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-19a15b52e167098b4804bf44: done: 0.222294 sec","job_status":"done","queue_duration_s":92.126183,"scheduling_latency_s":92.126246,"redis_calls":23,"redis_duration_s":0.006261,"redis_read_bytes":910,"redis_write_bytes":2540,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.003525,"redis_feature_flag_read_bytes":902,"redis_feature_flag_write_bytes":1375,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001531,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001205,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.121832,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.12049656,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.222294,"completed_at":"2026-11-13T10:40:07.859Z","load_balancing_strategy":"primary","db_duration_s":0.060756,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1833gitlab # [ 398.983585] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:07.862Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"87477e7909b0d0c3416f4165","created_at":"2026-11-13T10:38:35.528Z","trace_propagation_headers":{"sentry-trace":"d6df8a21b51d434499cababb3aba092d-d920e4f484b5402e","baggage":"sentry-trace_id=d6df8a21b51d434499cababb3aba092d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7SP20RP4HW876FQ6BD330","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:35.534Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-87477e7909b0d0c3416f4165: start","job_status":"start","queue_duration_s":92.328019,"scheduling_latency_s":92.328052}1834gitlab # [ 399.042934] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:07.991Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"87477e7909b0d0c3416f4165","created_at":"2026-11-13T10:38:35.528Z","trace_propagation_headers":{"sentry-trace":"d6df8a21b51d434499cababb3aba092d-d920e4f484b5402e","baggage":"sentry-trace_id=d6df8a21b51d434499cababb3aba092d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7SP20RP4HW876FQ6BD330","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:35.534Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-87477e7909b0d0c3416f4165: done: 0.129318 sec","job_status":"done","queue_duration_s":92.328019,"scheduling_latency_s":92.328052,"redis_calls":14,"redis_duration_s":0.003249,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000794,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001419,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001036,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.100476,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.09817767999999999,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.129318,"completed_at":"2026-11-13T10:40:07.991Z","load_balancing_strategy":"primary","db_duration_s":0.021724,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1835gitlab # [ 399.108301] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:07.993Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"feb19f85a66f10c72861cd2f","created_at":"2026-11-13T10:38:36.085Z","trace_propagation_headers":{"sentry-trace":"d6df8a21b51d434499cababb3aba092d-d920e4f484b5402e","baggage":"sentry-trace_id=d6df8a21b51d434499cababb3aba092d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7SP20RP4HW876FQ6BD330","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:36.092Z","job_size_bytes":5,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-feb19f85a66f10c72861cd2f: start","job_status":"start","queue_duration_s":91.901117,"scheduling_latency_s":91.901144}1836gitlab # [ 399.220319] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:08.168Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"feb19f85a66f10c72861cd2f","created_at":"2026-11-13T10:38:36.085Z","trace_propagation_headers":{"sentry-trace":"d6df8a21b51d434499cababb3aba092d-d920e4f484b5402e","baggage":"sentry-trace_id=d6df8a21b51d434499cababb3aba092d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7SP20RP4HW876FQ6BD330","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:36.092Z","job_size_bytes":5,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-feb19f85a66f10c72861cd2f: done: 0.175604 sec","job_status":"done","queue_duration_s":91.901117,"scheduling_latency_s":91.901144,"redis_calls":12,"redis_duration_s":0.002191,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000569,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001277,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000345,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.144279,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025786,"duration_s":0.175604,"completed_at":"2026-11-13T10:40:08.168Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.025827,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1837gitlab # [ 399.289820] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:08.170Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"ef358d9b8d5697b20b8d9446","created_at":"2026-11-13T10:38:36.123Z","trace_propagation_headers":{"sentry-trace":"d6df8a21b51d434499cababb3aba092d-d920e4f484b5402e","baggage":"sentry-trace_id=d6df8a21b51d434499cababb3aba092d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7SP20RP4HW876FQ6BD330","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:36.128Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-ef358d9b8d5697b20b8d9446: start","job_status":"start","queue_duration_s":92.041981,"scheduling_latency_s":92.042003}1838gitlab # [ 399.429503] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:08.378Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"ef358d9b8d5697b20b8d9446","created_at":"2026-11-13T10:38:36.123Z","trace_propagation_headers":{"sentry-trace":"d6df8a21b51d434499cababb3aba092d-d920e4f484b5402e","baggage":"sentry-trace_id=d6df8a21b51d434499cababb3aba092d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7SP20RP4HW876FQ6BD330","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:36.128Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-ef358d9b8d5697b20b8d9446: done: 0.207785 sec","job_status":"done","queue_duration_s":92.041981,"scheduling_latency_s":92.042003,"redis_calls":11,"redis_duration_s":0.002251,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000586,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001248,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000417,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.171,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.171,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.143006,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00039008,"duration_s":0.207785,"completed_at":"2026-11-13T10:40:08.378Z","load_balancing_strategy":"primary","db_duration_s":0.067552,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1839gitlab # [ 399.497472] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:08.379Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"c1e4b9b5d6a565fee608c4ea","created_at":"2026-11-13T10:38:36.132Z","trace_propagation_headers":{"sentry-trace":"d6df8a21b51d434499cababb3aba092d-d920e4f484b5402e","baggage":"sentry-trace_id=d6df8a21b51d434499cababb3aba092d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7SP20RP4HW876FQ6BD330","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:36.137Z","job_size_bytes":89,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-c1e4b9b5d6a565fee608c4ea: start","job_status":"start","queue_duration_s":92.242851,"scheduling_latency_s":92.242874}1840gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5'1841gitlab # [ 399.517893] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:08.417Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"c1e4b9b5d6a565fee608c4ea","created_at":"2026-11-13T10:38:36.132Z","trace_propagation_headers":{"sentry-trace":"d6df8a21b51d434499cababb3aba092d-d920e4f484b5402e","baggage":"sentry-trace_id=d6df8a21b51d434499cababb3aba092d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7SP20RP4HW876FQ6BD330","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:36.137Z","job_size_bytes":89,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-c1e4b9b5d6a565fee608c4ea: done: 0.037626 sec","job_status":"done","queue_duration_s":92.242851,"scheduling_latency_s":92.242874,"redis_calls":11,"redis_duration_s":0.002009,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000528,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001099,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000382,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025179,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00029678,"duration_s":0.037626,"completed_at":"2026-11-13T10:40:08.417Z","load_balancing_strategy":"primary","db_duration_s":0.005445,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1842gitlab # [ 399.565266] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:08.418Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 5c12ba2ba5cde5b1dd1bfa095c2839632222f25c refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"502a58cb9f5263405de72d01","created_at":"2026-11-13T10:38:43.202Z","trace_propagation_headers":{"sentry-trace":"1fa6eccd68f44a29905cb41224adbb6d-a5483f23ce4840ca","baggage":"sentry-trace_id=1fa6eccd68f44a29905cb41224adbb6d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7SY1QCQ856G1GYR454CPX","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:43.237Z","job_size_bytes":185,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-502a58cb9f5263405de72d01: start","job_status":"start","queue_duration_s":85.181847,"scheduling_latency_s":85.181868}1843gitlab # [ 399.582688] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:08.434Z","message":"changes: \"0000000000000000000000000000000000000000 5c12ba2ba5cde5b1dd1bfa095c2839632222f25c refs/heads/master\\n\""}1844gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5', in 0.04 seconds)1845gitlab # [ 399.585669] gitaly[905]: time="2026-11-13T10:40:08.471Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7SY1QCQ856G1GYR454CPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-13T10:40:18.010" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:40:08.460" grpc.time_ms=10.25 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1846gitlab # [ 399.599409] gitaly[905]: time="2026-11-13T10:40:08.485Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7SY1QCQ856G1GYR454CPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-13T10:40:18.026" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:40:08.477" grpc.time_ms=8.162 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1847gitlab # [ 399.613703] nixbot[3443]: {"ts": "2026-11-13T10:40:08.541Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:54956 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5 HTTP/1.1\" 200"}1848gitlab # [ 399.616560] gitaly[905]: time="2026-11-13T10:40:08.556Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7SY1QCQ856G1GYR454CPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-11-13T10:40:18.093" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:40:08.543" grpc.time_ms=12.166 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1849gitlab # [ 399.648940] gitaly[905]: time="2026-11-13T10:40:08.598Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7SY1QCQ856G1GYR454CPX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3936 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:40:08.597" grpc.time_ms=1.115 pid=905 span.kind=server system=grpc1850gitlab # [ 399.678109] gitaly[905]: time="2026-11-13T10:40:08.627Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7SY1QCQ856G1GYR454CPX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3938 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:40:08.626" grpc.time_ms=0.443 pid=905 span.kind=server system=grpc1851gitlab # [ 399.694090] gitaly[905]: time="2026-11-13T10:40:08.636Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7SY1QCQ856G1GYR454CPX diskcache=7c69a90f-efae-447d-9515-300804a76f81 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-13T10:40:18.117" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:40:08.567" pid=905 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1852gitlab # [ 399.709071] gitaly[905]: time="2026-11-13T10:40:08.636Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=73 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5095 command.oublock=8 command.real_time_ms=64 command.spawn_token_fork_ms=0 command.system_time_ms=36 command.user_time_ms=37 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7SY1QCQ856G1GYR454CPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-13T10:40:18.117" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:40:08.567" grpc.time_ms=68.958 pid=905 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1853gitlab # [ 399.726052] gitaly[905]: time="2026-11-13T10:40:08.651Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7SY1QCQ856G1GYR454CPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-13T10:40:18.200" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:40:08.650" grpc.time_ms=0.206 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1854gitlab # [ 399.738049] gitaly[905]: time="2026-11-13T10:40:08.669Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=862 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7SY1QCQ856G1GYR454CPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-11-13T10:40:18.205" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:40:08.656" grpc.time_ms=13.125 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1855gitlab # [ 399.832983] gitaly[905]: time="2026-11-13T10:40:08.782Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=313 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7SY1QCQ856G1GYR454CPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-11-13T10:40:38.075" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:08.775" grpc.time_ms=6.545 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1856gitlab # [ 399.866955] gitaly[905]: time="2026-11-13T10:40:08.816Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7SY1QCQ856G1GYR454CPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-11-13T10:40:18.005" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:40:08.815" grpc.time_ms=0.282 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1857gitlab # [ 399.884035] gitaly[905]: time="2026-11-13T10:40:08.824Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=316 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7SY1QCQ856G1GYR454CPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-11-13T10:40:18.008" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-11-13T10:40:08.818" grpc.time_ms=6.257 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1858gitlab # [ 399.904363] gitaly[905]: time="2026-11-13T10:40:08.848Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7SY1QCQ856G1GYR454CPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-13T10:40:18.029" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:40:08.839" grpc.time_ms=8.906 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1859gitlab # [ 399.921523] gitaly[905]: time="2026-11-13T10:40:08.863Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01M7A7SY1QCQ856G1GYR454CPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-13T10:40:18.044" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:40:08.854" grpc.time_ms=9.03 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1860gitlab # [ 399.942408] gitaly[905]: time="2026-11-13T10:40:08.891Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=883 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7SY1QCQ856G1GYR454CPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-11-13T10:40:38.176" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:08.877" grpc.time_ms=14.492 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1861gitlab # [ 400.499255] gitaly[905]: time="2026-11-13T10:40:09.448Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7SY1QCQ856G1GYR454CPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-11-13T10:40:19.006" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:09.446" grpc.time_ms=1.802 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1862gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5'1863gitlab # [ 400.542610] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:09.490Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 5c12ba2ba5cde5b1dd1bfa095c2839632222f25c refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"502a58cb9f5263405de72d01","created_at":"2026-11-13T10:38:43.202Z","trace_propagation_headers":{"sentry-trace":"1fa6eccd68f44a29905cb41224adbb6d-a5483f23ce4840ca","baggage":"sentry-trace_id=1fa6eccd68f44a29905cb41224adbb6d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7SY1QCQ856G1GYR454CPX","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:38:43.237Z","job_size_bytes":185,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-502a58cb9f5263405de72d01: done: 1.071638 sec","job_status":"done","queue_duration_s":85.181847,"scheduling_latency_s":85.181868,"gitaly_calls":13,"gitaly_duration_s":0.195519,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.040089,"redis_read_bytes":2103,"redis_write_bytes":21750,"redis_cache_calls":15,"redis_cache_duration_s":0.003671,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.005647,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.001507,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7838,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.004154,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.014804,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.010306,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.053,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.284,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.315,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.053,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.791161,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00057706,"exclusive_lock_hold_duration_s":0.02813224,"duration_s":1.071638,"completed_at":"2026-11-13T10:40:09.490Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.093157,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1864gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5', in 0.04 seconds)1865gitlab # [ 400.623204] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:09.492Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"0c9fda9540a359d73aec025b","created_at":"2026-11-13T10:38:48.535Z","trace_propagation_headers":{"sentry-trace":"2dfde065976e4074ae8481e69bd64a34-7559668aa4254867","baggage":"sentry-trace_id=2dfde065976e4074ae8481e69bd64a34,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7T3VBGCY888589SVN5M8J","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:48.543Z","job_size_bytes":65,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-0c9fda9540a359d73aec025b: start","job_status":"start","queue_duration_s":80.948923,"scheduling_latency_s":80.948954}1866gitlab # [ 400.646138] nixbot[3443]: {"ts": "2026-11-13T10:40:09.580Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:54968 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5 HTTP/1.1\" 200"}1867gitlab # [ 400.654019] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:09.602Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"0c9fda9540a359d73aec025b","created_at":"2026-11-13T10:38:48.535Z","trace_propagation_headers":{"sentry-trace":"2dfde065976e4074ae8481e69bd64a34-7559668aa4254867","baggage":"sentry-trace_id=2dfde065976e4074ae8481e69bd64a34,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7T3VBGCY888589SVN5M8J","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:48.543Z","job_size_bytes":65,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-0c9fda9540a359d73aec025b: done: 0.110225 sec","job_status":"done","queue_duration_s":80.948923,"scheduling_latency_s":80.948954,"redis_calls":16,"redis_duration_s":0.004325,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.002496,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001362,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000467,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.079932,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00045464,"duration_s":0.110225,"completed_at":"2026-11-13T10:40:09.602Z","load_balancing_strategy":"primary","db_duration_s":0.021984,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1868gitlab # [ 400.704797] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:09.604Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"4031f89e3a571194053cb68f","created_at":"2026-11-13T10:38:48.544Z","trace_propagation_headers":{"sentry-trace":"2dfde065976e4074ae8481e69bd64a34-7559668aa4254867","baggage":"sentry-trace_id=2dfde065976e4074ae8481e69bd64a34,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7T3VBGCY888589SVN5M8J","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:48.550Z","job_size_bytes":65,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-4031f89e3a571194053cb68f: start","job_status":"start","queue_duration_s":81.053896,"scheduling_latency_s":81.05392}1869gitlab # [ 400.767823] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:09.716Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"4031f89e3a571194053cb68f","created_at":"2026-11-13T10:38:48.544Z","trace_propagation_headers":{"sentry-trace":"2dfde065976e4074ae8481e69bd64a34-7559668aa4254867","baggage":"sentry-trace_id=2dfde065976e4074ae8481e69bd64a34,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7T3VBGCY888589SVN5M8J","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:48.550Z","job_size_bytes":65,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-4031f89e3a571194053cb68f: done: 0.111839 sec","job_status":"done","queue_duration_s":81.053896,"scheduling_latency_s":81.05392,"redis_calls":21,"redis_duration_s":0.004155,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.001792,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001214,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001149,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.074834,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003615,"duration_s":0.111839,"completed_at":"2026-11-13T10:40:09.716Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.028898,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1870gitlab # [ 400.831982] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:09.718Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"6e0c5faab7a8c52685f59e90","created_at":"2026-11-13T10:38:48.691Z","trace_propagation_headers":{"sentry-trace":"2dfde065976e4074ae8481e69bd64a34-7559668aa4254867","baggage":"sentry-trace_id=2dfde065976e4074ae8481e69bd64a34,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7T3VBGCY888589SVN5M8J","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:48.696Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-6e0c5faab7a8c52685f59e90: start","job_status":"start","queue_duration_s":81.021459,"scheduling_latency_s":81.021482}1871gitlab # [ 401.063150] gitaly[905]: time="2026-11-13T10:40:10.011Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7T3VBGCY888589SVN5M8J grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3897 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:40:10.011" grpc.time_ms=0.491 pid=905 span.kind=server system=grpc1872gitlab # [ 401.097922] gitaly[905]: time="2026-11-13T10:40:10.047Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7T3VBGCY888589SVN5M8J grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3895 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:40:10.046" grpc.time_ms=0.547 pid=905 span.kind=server system=grpc1873gitlab # [ 401.129859] gitaly[905]: time="2026-11-13T10:40:10.079Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7T3VBGCY888589SVN5M8J grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3897 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:40:10.078" grpc.time_ms=0.407 pid=905 span.kind=server system=grpc1874gitlab # [ 401.145683] gitaly[905]: time="2026-11-13T10:40:10.087Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7T3VBGCY888589SVN5M8J diskcache=75287a0c-df1d-47ed-aa5a-6a5bcb366129 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-13T10:40:39.278" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:40:09.979" pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1875gitlab # [ 401.161191] gitaly[905]: time="2026-11-13T10:40:10.088Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=112 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7556 command.oublock=0 command.real_time_ms=104 command.spawn_token_fork_ms=0 command.system_time_ms=54 command.user_time_ms=58 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7T3VBGCY888589SVN5M8J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-13T10:40:39.278" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:40:09.979" grpc.time_ms=108.822 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1876gitlab # [ 401.272291] gitaly[905]: time="2026-11-13T10:40:10.221Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7T3VBGCY888589SVN5M8J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-13T10:40:40.019" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:10.220" grpc.time_ms=1.148 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1877gitlab # [ 401.310231] gitaly[905]: time="2026-11-13T10:40:10.259Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7T3VBGCY888589SVN5M8J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:40:40.057" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:10.258" grpc.time_ms=0.92 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1878gitlab # [ 401.330362] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:10.276Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"6e0c5faab7a8c52685f59e90","created_at":"2026-11-13T10:38:48.691Z","trace_propagation_headers":{"sentry-trace":"2dfde065976e4074ae8481e69bd64a34-7559668aa4254867","baggage":"sentry-trace_id=2dfde065976e4074ae8481e69bd64a34,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7T3VBGCY888589SVN5M8J","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-13T10:38:48.696Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-6e0c5faab7a8c52685f59e90: done: 0.558334 sec","job_status":"done","queue_duration_s":81.021459,"scheduling_latency_s":81.021482,"gitaly_calls":3,"gitaly_duration_s":0.117978,"redis_calls":117,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.022082,"redis_read_bytes":455,"redis_write_bytes":28000,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001102,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001128,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.007816,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":25,"redis_queues_duration_s":0.003778,"redis_queues_read_bytes":34,"redis_queues_write_bytes":14494,"redis_queues_metadata_calls":21,"redis_queues_metadata_duration_s":0.004458,"redis_queues_metadata_read_bytes":32,"redis_queues_metadata_write_bytes":3370,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000534,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000494,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00167,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.176,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.023,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.027,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.146,"db_ci_duration_s":0.03,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.367033,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.38019596,"duration_s":0.558334,"completed_at":"2026-11-13T10:40:10.276Z","load_balancing_strategy":"primary","db_duration_s":0.18686,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1879gitlab # [ 401.400596] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:10.278Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","5c12ba2ba5cde5b1dd1bfa095c2839632222f25c 6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"a1ccb75e4b6362a768b7ee53","created_at":"2026-11-13T10:40:06.204Z","trace_propagation_headers":{"sentry-trace":"597dbc31f9ab46c4a245d75aa6dcd926-8f9fc9ab5fd7493c","baggage":"sentry-trace_id=597dbc31f9ab46c4a245d75aa6dcd926,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7WFGXP3SWPPDZ22MN1RGB","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:06.214Z","job_size_bytes":185,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-a1ccb75e4b6362a768b7ee53: start","job_status":"start","queue_duration_s":4.063671,"scheduling_latency_s":4.063691}1880gitlab # [ 401.417976] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:10.290Z","message":"changes: \"5c12ba2ba5cde5b1dd1bfa095c2839632222f25c 6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5 refs/heads/master\\n\""}1881gitlab # [ 401.420432] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:10.335Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"bced2dd1154dd76e85124ff9","created_at":"2026-11-13T10:40:10.333Z","trace_propagation_headers":{"sentry-trace":"597dbc31f9ab46c4a245d75aa6dcd926-bf0636f0bb8f4dd1","baggage":"sentry-trace_id=597dbc31f9ab46c4a245d75aa6dcd926,sentry-release=Unknown"},"correlation_id":"01M7A7WFGXP3SWPPDZ22MN1RGB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"052bbfd9732cefaa89b66c3c","job_size_bytes":46,"pid":3321,"job_status":"deduplicated","message":"ProjectCacheWorker JID-bced2dd1154dd76e85124ff9: deduplicated: until executing","deduplication.type":"until executing"}1882gitlab # [ 401.435787] gitaly[905]: time="2026-11-13T10:40:10.317Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7WFGXP3SWPPDZ22MN1RGB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-13T10:40:20.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:40:10.306" grpc.time_ms=10.416 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1883gitlab # [ 401.449485] gitaly[905]: time="2026-11-13T10:40:10.330Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WFGXP3SWPPDZ22MN1RGB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-13T10:40:20.022" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:40:10.322" grpc.time_ms=7.962 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1884gitlab # [ 401.463161] gitaly[905]: time="2026-11-13T10:40:10.362Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7WFGXP3SWPPDZ22MN1RGB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-11-13T10:40:20.053" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:40:10.353" grpc.time_ms=8.491 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1885gitlab # [ 401.476869] gitaly[905]: time="2026-11-13T10:40:10.376Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=598 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7WFGXP3SWPPDZ22MN1RGB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-11-13T10:40:40.166" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:10.366" grpc.time_ms=9.929 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1886gitlab # [ 401.492449] gitaly[905]: time="2026-11-13T10:40:10.393Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=318 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7WFGXP3SWPPDZ22MN1RGB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-11-13T10:40:20.088" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-11-13T10:40:10.388" grpc.time_ms=5.345 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1887gitlab # [ 401.506169] gitaly[905]: time="2026-11-13T10:40:10.413Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7WFGXP3SWPPDZ22MN1RGB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-13T10:40:20.103" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:40:10.404" grpc.time_ms=9.718 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1888gitlab # [ 401.562320] gitaly[905]: time="2026-11-13T10:40:10.511Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7WFGXP3SWPPDZ22MN1RGB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-11-13T10:40:20.209" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:10.510" grpc.time_ms=1.21 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1889gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5'1890gitlab # [ 401.584901] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:10.533Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","5c12ba2ba5cde5b1dd1bfa095c2839632222f25c 6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"a1ccb75e4b6362a768b7ee53","created_at":"2026-11-13T10:40:06.204Z","trace_propagation_headers":{"sentry-trace":"597dbc31f9ab46c4a245d75aa6dcd926-8f9fc9ab5fd7493c","baggage":"sentry-trace_id=597dbc31f9ab46c4a245d75aa6dcd926,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7WFGXP3SWPPDZ22MN1RGB","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:06.214Z","job_size_bytes":185,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-a1ccb75e4b6362a768b7ee53: done: 0.25505 sec","job_status":"done","queue_duration_s":4.063671,"scheduling_latency_s":4.063691,"gitaly_calls":7,"gitaly_duration_s":0.0696,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.018795,"redis_read_bytes":501,"redis_write_bytes":11942,"redis_cache_calls":9,"redis_cache_duration_s":0.001958,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.000912,"redis_queues_read_bytes":9,"redis_queues_write_bytes":5719,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.003557,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.005804,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.006564,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.024,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.024,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.15874,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00059074,"exclusive_lock_hold_duration_s":0.0196995,"duration_s":0.25505,"completed_at":"2026-11-13T10:40:10.533Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.027125,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1891gitlab # [ 401.648228] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:10.534Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"052bbfd9732cefaa89b66c3c","created_at":"2026-11-13T10:40:08.488Z","trace_propagation_headers":{"sentry-trace":"1fa6eccd68f44a29905cb41224adbb6d-ac3d02e105c241b3","baggage":"sentry-trace_id=1fa6eccd68f44a29905cb41224adbb6d,sentry-release=Unknown"},"correlation_id":"01M7A7SY1QCQ856G1GYR454CPX","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:08.511Z","job_size_bytes":46,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-052bbfd9732cefaa89b66c3c: start","job_status":"start","queue_duration_s":2.023122,"scheduling_latency_s":2.023143}1892gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5', in 0.03 seconds)1893gitlab # [ 401.665139] gitaly[905]: time="2026-11-13T10:40:10.586Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7SY1QCQ856G1GYR454CPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-11-13T16:40:10.580" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:40:10.580" grpc.time_ms=5.151 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1894gitlab # [ 401.676468] gitaly[905]: time="2026-11-13T10:40:10.606Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=597 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7SY1QCQ856G1GYR454CPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-11-13T10:40:40.396" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:10.597" grpc.time_ms=9.197 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1895gitlab # [ 401.690208] nixbot[3443]: {"ts": "2026-11-13T10:40:10.614Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:47642 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5 HTTP/1.1\" 200"}1896gitlab # [ 401.693114] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:10.639Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"052bbfd9732cefaa89b66c3c","created_at":"2026-11-13T10:40:08.488Z","trace_propagation_headers":{"sentry-trace":"1fa6eccd68f44a29905cb41224adbb6d-ac3d02e105c241b3","baggage":"sentry-trace_id=1fa6eccd68f44a29905cb41224adbb6d,sentry-release=Unknown"},"correlation_id":"01M7A7SY1QCQ856G1GYR454CPX","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:08.511Z","job_size_bytes":46,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-052bbfd9732cefaa89b66c3c: done: 0.104905 sec","job_status":"done","queue_duration_s":2.023122,"scheduling_latency_s":2.023143,"gitaly_calls":2,"gitaly_duration_s":0.019239,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.006048,"redis_read_bytes":69,"redis_write_bytes":4887,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001243,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000559,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2115,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001482,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.002173,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":753,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000591,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.067025,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00032222,"duration_s":0.104905,"completed_at":"2026-11-13T10:40:10.639Z","load_balancing_strategy":"primary","db_duration_s":0.011028,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1897gitlab # [ 401.742870] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:10.641Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"8ad0852bf785891bc8dc7029","created_at":"2026-11-13T10:40:08.531Z","trace_propagation_headers":{"sentry-trace":"1fa6eccd68f44a29905cb41224adbb6d-ac3d02e105c241b3","baggage":"sentry-trace_id=1fa6eccd68f44a29905cb41224adbb6d,sentry-release=Unknown"},"correlation_id":"01M7A7SY1QCQ856G1GYR454CPX","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:08.538Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-8ad0852bf785891bc8dc7029: start","job_status":"start","queue_duration_s":2.102192,"scheduling_latency_s":2.102215}1898gitlab # [ 401.760030] gitaly[905]: time="2026-11-13T10:40:10.688Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7SY1QCQ856G1GYR454CPX diskcache=7cb221e0-36b0-4ac9-9d61-5c95789c7613 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-11-13T16:40:10.667" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:10.667" pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1899gitlab # [ 401.770099] gitaly[905]: time="2026-11-13T10:40:10.688Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1174 command.oublock=0 command.real_time_ms=24 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=10 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7SY1QCQ856G1GYR454CPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-11-13T16:40:10.667" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:10.667" grpc.time_ms=20.705 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1900gitlab # [ 401.785625] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:10.728Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"8ad0852bf785891bc8dc7029","created_at":"2026-11-13T10:40:08.531Z","trace_propagation_headers":{"sentry-trace":"1fa6eccd68f44a29905cb41224adbb6d-ac3d02e105c241b3","baggage":"sentry-trace_id=1fa6eccd68f44a29905cb41224adbb6d,sentry-release=Unknown"},"correlation_id":"01M7A7SY1QCQ856G1GYR454CPX","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:08.538Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-8ad0852bf785891bc8dc7029: done: 0.087409 sec","job_status":"done","queue_duration_s":2.102192,"scheduling_latency_s":2.102215,"gitaly_calls":1,"gitaly_duration_s":0.023107,"redis_calls":15,"redis_duration_s":0.002862,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000391,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001059,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00047,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000942,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.011,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.021,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.044939,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.06033012,"duration_s":0.087409,"completed_at":"2026-11-13T10:40:10.728Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01448,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1901gitlab # [ 401.834376] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:10.729Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"2b578cf3dc30f9cdde1e2e71","created_at":"2026-11-13T10:40:08.646Z","trace_propagation_headers":{"sentry-trace":"1fa6eccd68f44a29905cb41224adbb6d-ac3d02e105c241b3","baggage":"sentry-trace_id=1fa6eccd68f44a29905cb41224adbb6d,sentry-release=Unknown"},"correlation_id":"01M7A7SY1QCQ856G1GYR454CPX","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:08.648Z","job_size_bytes":23,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-2b578cf3dc30f9cdde1e2e71: start","job_status":"start","queue_duration_s":2.081521,"scheduling_latency_s":2.081541}1902gitlab # [ 401.850228] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:10.774Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"2b578cf3dc30f9cdde1e2e71","created_at":"2026-11-13T10:40:08.646Z","trace_propagation_headers":{"sentry-trace":"1fa6eccd68f44a29905cb41224adbb6d-ac3d02e105c241b3","baggage":"sentry-trace_id=1fa6eccd68f44a29905cb41224adbb6d,sentry-release=Unknown"},"correlation_id":"01M7A7SY1QCQ856G1GYR454CPX","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:08.648Z","job_size_bytes":23,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-2b578cf3dc30f9cdde1e2e71: done: 0.04511 sec","job_status":"done","queue_duration_s":2.081521,"scheduling_latency_s":2.081541,"gitaly_calls":1,"gitaly_duration_s":0.011044,"redis_calls":16,"redis_duration_s":0.002941,"redis_read_bytes":52,"redis_write_bytes":2048,"redis_queues_calls":1,"redis_queues_duration_s":0.000256,"redis_queues_read_bytes":1,"redis_queues_write_bytes":979,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000847,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.00127,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000568,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027024,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.04511,"completed_at":"2026-11-13T10:40:10.774Z","load_balancing_strategy":"primary","db_duration_s":0.003023,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1903gitlab # [ 401.896437] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:10.776Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"b300bdd4007985b75cbe768a","created_at":"2026-11-13T10:40:09.246Z","trace_propagation_headers":{"sentry-trace":"1fa6eccd68f44a29905cb41224adbb6d-ac3d02e105c241b3","baggage":"sentry-trace_id=1fa6eccd68f44a29905cb41224adbb6d,sentry-release=Unknown"},"correlation_id":"01M7A7SY1QCQ856G1GYR454CPX","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:27e4968feb22434cc4d283982075ca2c753389b36f0d0a7378a54b7c0ff0e411","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:09.253Z","job_size_bytes":467,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-b300bdd4007985b75cbe768a: start","job_status":"start","queue_duration_s":1.522905,"scheduling_latency_s":1.522922}1904gitlab # [ 401.914545] gitaly[905]: time="2026-11-13T10:40:10.766Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=595 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7SY1QCQ856G1GYR454CPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-11-13T10:40:40.556" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:10.757" grpc.time_ms=9.066 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1905gitlab # [ 401.928111] gitaly[905]: time="2026-11-13T10:40:10.802Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7SY1QCQ856G1GYR454CPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:40:40.600" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:10.801" grpc.time_ms=0.654 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1906gitlab # [ 401.940742] gitaly[905]: time="2026-11-13T10:40:10.812Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7SY1QCQ856G1GYR454CPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-11-13T10:40:20.004" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:40:10.804" grpc.time_ms=8.529 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1907gitlab # [ 401.977524] gitaly[905]: time="2026-11-13T10:40:10.926Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7SY1QCQ856G1GYR454CPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-13T10:40:40.725" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:10.926" grpc.time_ms=0.704 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1908gitlab # [ 401.990754] gitaly[905]: time="2026-11-13T10:40:10.933Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=50 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7SY1QCQ856G1GYR454CPX error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-11-13T10:40:40.731" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:10.931" grpc.time_ms=2.383 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1909gitlab # [ 402.103692] gitaly[905]: time="2026-11-13T10:40:11.052Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7SY1QCQ856G1GYR454CPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-13T10:40:21.001" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:40:11.042" grpc.time_ms=10.143 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1910gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5'1911gitlab # [ 402.659088] gitaly[905]: time="2026-11-13T10:40:11.608Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=881 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M7A7SY1QCQ856G1GYR454CPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-11-13T10:40:41.391" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:11.591" grpc.time_ms=16.809 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1912gitlab # [ 402.683056] gitaly[905]: time="2026-11-13T10:40:11.625Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7SY1QCQ856G1GYR454CPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-11-13T10:40:41.410" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:11.611" grpc.time_ms=13.842 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1913gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5', in 0.05 seconds)1914gitlab # [ 402.713504] nixbot[3443]: {"ts": "2026-11-13T10:40:11.661Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:47650 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5 HTTP/1.1\" 200"}1915gitlab # [ 402.718612] sidekiq-cluster[3321]: {"severity":"WARN","time":"2026-11-13T10:40:11.663Z","class":"CreatePipelineWorker","correlation_id":"01M7A7SY1QCQ856G1GYR454CPX","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"5c12ba2ba5cde5b1dd1bfa095c2839632222f25c","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"5c12ba2ba5cde5b1dd1bfa095c2839632222f25c","ref":"refs/heads/master"},"retry":0}1916gitlab # [ 402.730127] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:11.671Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"b300bdd4007985b75cbe768a","created_at":"2026-11-13T10:40:09.246Z","trace_propagation_headers":{"sentry-trace":"1fa6eccd68f44a29905cb41224adbb6d-ac3d02e105c241b3","baggage":"sentry-trace_id=1fa6eccd68f44a29905cb41224adbb6d,sentry-release=Unknown"},"correlation_id":"01M7A7SY1QCQ856G1GYR454CPX","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:27e4968feb22434cc4d283982075ca2c753389b36f0d0a7378a54b7c0ff0e411","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:09.253Z","job_size_bytes":467,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-b300bdd4007985b75cbe768a: done: 0.895234 sec","job_status":"done","queue_duration_s":1.522905,"scheduling_latency_s":1.522922,"gitaly_calls":7,"gitaly_duration_s":0.068632,"redis_calls":51,"redis_duration_s":0.008432,"redis_read_bytes":169,"redis_write_bytes":4168,"redis_cache_calls":4,"redis_cache_duration_s":0.000797,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.003335,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001045,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002276,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000979,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.775135,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002289,"duration_s":0.895234,"completed_at":"2026-11-13T10:40:11.671Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.052935,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1917gitlab # [ 402.795702] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:11.673Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"600f68d2-d249-4eb5-9447-b2e9241f5327\"}"],"class":"WebHookWorker","jid":"24e8e3a419bcfa9d03a9208e","created_at":"2026-11-13T10:40:09.414Z","trace_propagation_headers":{"sentry-trace":"1fa6eccd68f44a29905cb41224adbb6d-ac3d02e105c241b3","baggage":"sentry-trace_id=1fa6eccd68f44a29905cb41224adbb6d,sentry-release=Unknown"},"correlation_id":"01M7A7SY1QCQ856G1GYR454CPX","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7226715edfd0e36c8394ecd78f524d2608e5b81176c6548babb0b38b31a3ca07","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:09.419Z","job_size_bytes":1751,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-24e8e3a419bcfa9d03a9208e: start","job_status":"start","queue_duration_s":2.2534,"scheduling_latency_s":2.253422}1918gitlab # [ 402.869425] nixbot[3443]: {"ts": "2026-11-13T10:40:11.818Z", "level": "info", "logger": "nixbot.webhooks", "msg": "webhook accepted", "delivery": "0836c78e-b792-4cb9-bda8-6052536f91b6", "event": "ChangeRequest", "forge": "gitlab", "repo_id": "1", "branch": "master", "sha": "5c12ba2ba5cde5b1dd1bfa095c2839632222f25c", "pr": null}1919gitlab # [ 402.875882] nixbot[3443]: {"ts": "2026-11-13T10:40:11.825Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:47664 - \"POST /webhooks/gitlab HTTP/1.1\" 202"}1920gitlab # [ 402.900895] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:11.849Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"600f68d2-d249-4eb5-9447-b2e9241f5327\"}"],"class":"WebHookWorker","jid":"24e8e3a419bcfa9d03a9208e","created_at":"2026-11-13T10:40:09.414Z","trace_propagation_headers":{"sentry-trace":"1fa6eccd68f44a29905cb41224adbb6d-ac3d02e105c241b3","baggage":"sentry-trace_id=1fa6eccd68f44a29905cb41224adbb6d,sentry-release=Unknown"},"correlation_id":"01M7A7SY1QCQ856G1GYR454CPX","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7226715edfd0e36c8394ecd78f524d2608e5b81176c6548babb0b38b31a3ca07","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:09.419Z","job_size_bytes":1751,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-24e8e3a419bcfa9d03a9208e: done: 0.17607 sec","job_status":"done","queue_duration_s":2.2534,"scheduling_latency_s":2.253422,"redis_calls":19,"redis_duration_s":0.004162,"redis_read_bytes":32,"redis_write_bytes":5363,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001103,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.00043,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3725,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001662,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000967,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.0702746200000206,"cpu_s":0.091818,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00040364,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.17607,"completed_at":"2026-11-13T10:40:11.849Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003748,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1921gitlab # [ 402.972123] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:11.851Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"86ce84aee488096235edb996","created_at":"2026-11-13T10:40:09.963Z","trace_propagation_headers":{"sentry-trace":"2dfde065976e4074ae8481e69bd64a34-a5cb3fd4064a458a","baggage":"sentry-trace_id=2dfde065976e4074ae8481e69bd64a34,sentry-release=Unknown"},"correlation_id":"01M7A7T3VBGCY888589SVN5M8J","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:09.968Z","job_size_bytes":23,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-86ce84aee488096235edb996: start","job_status":"start","queue_duration_s":1.883319,"scheduling_latency_s":1.883346}1922gitlab # [ 402.989741] gitaly[905]: time="2026-11-13T10:40:11.936Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7WNJEP524M3A1Q3G7C8YZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:40:11.927" grpc.time_ms=9.033 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1923gitlab # [ 403.003624] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:40:11 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 831924gitlab # [ 403.018018] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:11.966Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"86ce84aee488096235edb996","created_at":"2026-11-13T10:40:09.963Z","trace_propagation_headers":{"sentry-trace":"2dfde065976e4074ae8481e69bd64a34-a5cb3fd4064a458a","baggage":"sentry-trace_id=2dfde065976e4074ae8481e69bd64a34,sentry-release=Unknown"},"correlation_id":"01M7A7T3VBGCY888589SVN5M8J","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:09.968Z","job_size_bytes":23,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-86ce84aee488096235edb996: done: 0.115263 sec","job_status":"done","queue_duration_s":1.883319,"scheduling_latency_s":1.883346,"redis_calls":10,"redis_duration_s":0.002393,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000579,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001305,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000509,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.089574,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00032092,"duration_s":0.115263,"completed_at":"2026-11-13T10:40:11.966Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.019424,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1925gitlab # [ 403.074903] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:11.968Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"cd481a0273eabacf6dca290c","created_at":"2026-11-13T10:40:10.094Z","trace_propagation_headers":{"sentry-trace":"2dfde065976e4074ae8481e69bd64a34-a5cb3fd4064a458a","baggage":"sentry-trace_id=2dfde065976e4074ae8481e69bd64a34,sentry-release=Unknown"},"correlation_id":"01M7A7T3VBGCY888589SVN5M8J","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:10.100Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-cd481a0273eabacf6dca290c: start","job_status":"start","queue_duration_s":1.867693,"scheduling_latency_s":1.867715}1926gitlab # [ 403.092798] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:12.007Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"cd481a0273eabacf6dca290c","created_at":"2026-11-13T10:40:10.094Z","trace_propagation_headers":{"sentry-trace":"2dfde065976e4074ae8481e69bd64a34-a5cb3fd4064a458a","baggage":"sentry-trace_id=2dfde065976e4074ae8481e69bd64a34,sentry-release=Unknown"},"correlation_id":"01M7A7T3VBGCY888589SVN5M8J","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:10.100Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-cd481a0273eabacf6dca290c: done: 0.039106 sec","job_status":"done","queue_duration_s":1.867693,"scheduling_latency_s":1.867715,"redis_calls":10,"redis_duration_s":0.002072,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000387,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001232,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000453,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025585,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00037092,"duration_s":0.039106,"completed_at":"2026-11-13T10:40:12.007Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005817,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1927gitlab # [ 403.138537] gitlab-workhorse[818]: time="2026-11-13T10:40:12Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671928gitlab # [ 403.140438] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:40:12 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 741929gitlab # [ 403.142650] gitaly[905]: time="2026-11-13T10:40:12.013Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WNN3A6YNVVR84ZPK8E4Q grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:40:12.002" pid=905 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1930gitlab # [ 403.154144] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:12.009Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"4b7478e98f9af9a112024e4c","created_at":"2026-11-13T10:40:10.118Z","trace_propagation_headers":{"sentry-trace":"2dfde065976e4074ae8481e69bd64a34-a5cb3fd4064a458a","baggage":"sentry-trace_id=2dfde065976e4074ae8481e69bd64a34,sentry-release=Unknown"},"correlation_id":"01M7A7T3VBGCY888589SVN5M8J","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:10.125Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-4b7478e98f9af9a112024e4c: start","job_status":"start","queue_duration_s":1.884506,"scheduling_latency_s":1.884527}1931gitlab # [ 403.170485] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:12.049Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"4b7478e98f9af9a112024e4c","created_at":"2026-11-13T10:40:10.118Z","trace_propagation_headers":{"sentry-trace":"2dfde065976e4074ae8481e69bd64a34-a5cb3fd4064a458a","baggage":"sentry-trace_id=2dfde065976e4074ae8481e69bd64a34,sentry-release=Unknown"},"correlation_id":"01M7A7T3VBGCY888589SVN5M8J","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:10.125Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-4b7478e98f9af9a112024e4c: done: 0.03998 sec","job_status":"done","queue_duration_s":1.884506,"scheduling_latency_s":1.884527,"redis_calls":10,"redis_duration_s":0.003668,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000824,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002505,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000339,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025667,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00021794,"duration_s":0.03998,"completed_at":"2026-11-13T10:40:12.049Z","load_balancing_strategy":"primary","db_duration_s":0.006081,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1932gitlab # [ 403.215355] gitlab-workhorse[818]: time="2026-11-13T10:40:12Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3411933gitlab # [ 403.217399] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:40:12 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 341 "" "git/2.55.0" 741934gitlab # [ 403.219316] gitaly[905]: time="2026-11-13T10:40:12.013Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WNN3A6YNVVR84ZPK8E4Q grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:40:12.002" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1935gitlab # [ 403.233325] nixbot[3443]: {"ts": "2026-11-13T10:40:12.127Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"}1936gitlab # [ 403.235761] nixbot[3443]: {"ts": "2026-11-13T10:40:12.128Z", "level": "info", "logger": "nixbot.after_build", "msg": "reused build: replaying effect statuses", "build_id": 1, "sha": "5c12ba2ba5cde5b1dd1bfa095c2839632222f25c"}1937gitlab # [ 403.238443] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:12.052Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"8c5c8ae78db3fac906dcaf22","created_at":"2026-11-13T10:40:10.142Z","trace_propagation_headers":{"sentry-trace":"2dfde065976e4074ae8481e69bd64a34-a5cb3fd4064a458a","baggage":"sentry-trace_id=2dfde065976e4074ae8481e69bd64a34,sentry-release=Unknown"},"correlation_id":"01M7A7T3VBGCY888589SVN5M8J","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:10.148Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-8c5c8ae78db3fac906dcaf22: start","job_status":"start","queue_duration_s":1.90481,"scheduling_latency_s":1.904829}1938gitlab # [ 403.255393] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:12.093Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"8c5c8ae78db3fac906dcaf22","created_at":"2026-11-13T10:40:10.142Z","trace_propagation_headers":{"sentry-trace":"2dfde065976e4074ae8481e69bd64a34-a5cb3fd4064a458a","baggage":"sentry-trace_id=2dfde065976e4074ae8481e69bd64a34,sentry-release=Unknown"},"correlation_id":"01M7A7T3VBGCY888589SVN5M8J","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:10.148Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-8c5c8ae78db3fac906dcaf22: done: 0.040498 sec","job_status":"done","queue_duration_s":1.90481,"scheduling_latency_s":1.904829,"redis_calls":13,"redis_duration_s":0.00343,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000693,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001643,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000384,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00071,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026172,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00046944,"duration_s":0.040498,"completed_at":"2026-11-13T10:40:12.093Z","load_balancing_strategy":"primary","db_duration_s":0.006158,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1939gitlab # [ 403.301888] gitaly[905]: time="2026-11-13T10:40:12.013Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WNN3A6YNVVR84ZPK8E4Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:40:12.002" grpc.time_ms=11.247 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1940gitlab # [ 403.316783] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:12.097Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"dfc8f464b6b13f50e5e8acfd","created_at":"2026-11-13T10:40:10.148Z","trace_propagation_headers":{"sentry-trace":"2dfde065976e4074ae8481e69bd64a34-a5cb3fd4064a458a","baggage":"sentry-trace_id=2dfde065976e4074ae8481e69bd64a34,sentry-release=Unknown"},"correlation_id":"01M7A7T3VBGCY888589SVN5M8J","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:10.154Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-dfc8f464b6b13f50e5e8acfd: start","job_status":"start","queue_duration_s":1.943002,"scheduling_latency_s":1.943023}1941gitlab # [ 403.333680] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:12.133Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"dfc8f464b6b13f50e5e8acfd","created_at":"2026-11-13T10:40:10.148Z","trace_propagation_headers":{"sentry-trace":"2dfde065976e4074ae8481e69bd64a34-a5cb3fd4064a458a","baggage":"sentry-trace_id=2dfde065976e4074ae8481e69bd64a34,sentry-release=Unknown"},"correlation_id":"01M7A7T3VBGCY888589SVN5M8J","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:10.154Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-dfc8f464b6b13f50e5e8acfd: done: 0.036346 sec","job_status":"done","queue_duration_s":1.943002,"scheduling_latency_s":1.943023,"redis_calls":10,"redis_duration_s":0.002282,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000378,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001139,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000765,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02418,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00021878,"duration_s":0.036346,"completed_at":"2026-11-13T10:40:12.133Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005822,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1942gitlab # [ 403.379422] gitaly[905]: time="2026-11-13T10:40:12.089Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WNQFJRY1HZNTTQACZ3AV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:40:12.069" pid=905 remote_ip="2001:db8:1::1" request_sha=6116ffb452f49c91fece27ef1c4dde5a3fecf4bc response_bytes=341 span.kind=server system=grpc1943gitlab # [ 403.390130] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:12.135Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"f776ccc447a1094ab1a5bfb5","created_at":"2026-11-13T10:40:10.155Z","trace_propagation_headers":{"sentry-trace":"2dfde065976e4074ae8481e69bd64a34-a5cb3fd4064a458a","baggage":"sentry-trace_id=2dfde065976e4074ae8481e69bd64a34,sentry-release=Unknown"},"correlation_id":"01M7A7T3VBGCY888589SVN5M8J","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:10.162Z","job_size_bytes":19,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-f776ccc447a1094ab1a5bfb5: start","job_status":"start","queue_duration_s":1.973045,"scheduling_latency_s":1.973062}1944gitlab # [ 403.406827] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:12.173Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"f776ccc447a1094ab1a5bfb5","created_at":"2026-11-13T10:40:10.155Z","trace_propagation_headers":{"sentry-trace":"2dfde065976e4074ae8481e69bd64a34-a5cb3fd4064a458a","baggage":"sentry-trace_id=2dfde065976e4074ae8481e69bd64a34,sentry-release=Unknown"},"correlation_id":"01M7A7T3VBGCY888589SVN5M8J","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:10.162Z","job_size_bytes":19,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-f776ccc447a1094ab1a5bfb5: done: 0.038261 sec","job_status":"done","queue_duration_s":1.973045,"scheduling_latency_s":1.973062,"redis_calls":11,"redis_duration_s":0.00876,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003796,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002127,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002837,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020487,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00036858,"duration_s":0.038261,"completed_at":"2026-11-13T10:40:12.173Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004663,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1945gitlab # [ 403.453107] gitaly[905]: time="2026-11-13T10:40:12.089Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1128 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=4 command.system_time_ms=7 command.user_time_ms=7 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WNQFJRY1HZNTTQACZ3AV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:40:12.069" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1946gitlab # [ 403.467155] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:40:12 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/5c12ba2ba5cde5b1dd1bfa095c2839632222f25c HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 2081947gitlab # [ 403.469821] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:12.175Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"9f13ce17bce84374f865070b","created_at":"2026-11-13T10:40:10.165Z","trace_propagation_headers":{"sentry-trace":"2dfde065976e4074ae8481e69bd64a34-a5cb3fd4064a458a","baggage":"sentry-trace_id=2dfde065976e4074ae8481e69bd64a34,sentry-release=Unknown"},"correlation_id":"01M7A7T3VBGCY888589SVN5M8J","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:10.170Z","job_size_bytes":66,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-9f13ce17bce84374f865070b: start","job_status":"start","queue_duration_s":2.005073,"scheduling_latency_s":2.005092}1948gitlab # [ 403.487644] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:12.217Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"9f13ce17bce84374f865070b","created_at":"2026-11-13T10:40:10.165Z","trace_propagation_headers":{"sentry-trace":"2dfde065976e4074ae8481e69bd64a34-a5cb3fd4064a458a","baggage":"sentry-trace_id=2dfde065976e4074ae8481e69bd64a34,sentry-release=Unknown"},"correlation_id":"01M7A7T3VBGCY888589SVN5M8J","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:10.170Z","job_size_bytes":66,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-9f13ce17bce84374f865070b: done: 0.041834 sec","job_status":"done","queue_duration_s":2.005073,"scheduling_latency_s":2.005092,"redis_calls":10,"redis_duration_s":0.009192,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002966,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005815,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000411,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022767,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00049496,"duration_s":0.041834,"completed_at":"2026-11-13T10:40:12.217Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005752,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1949gitlab # [ 403.534591] gitaly[905]: time="2026-11-13T10:40:12.090Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1128 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=4 command.system_time_ms=7 command.user_time_ms=7 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WNQFJRY1HZNTTQACZ3AV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:40:12.069" grpc.time_ms=20.875 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1950gitlab # [ 403.549985] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:12.218Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"4f3ddc729b725845f483f2b2","created_at":"2026-11-13T10:40:10.171Z","trace_propagation_headers":{"sentry-trace":"2dfde065976e4074ae8481e69bd64a34-a5cb3fd4064a458a","baggage":"sentry-trace_id=2dfde065976e4074ae8481e69bd64a34,sentry-release=Unknown"},"correlation_id":"01M7A7T3VBGCY888589SVN5M8J","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:10.175Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-4f3ddc729b725845f483f2b2: start","job_status":"start","queue_duration_s":2.042369,"scheduling_latency_s":2.042386}1951gitlab # [ 403.566546] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:12.264Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"4f3ddc729b725845f483f2b2","created_at":"2026-11-13T10:40:10.171Z","trace_propagation_headers":{"sentry-trace":"2dfde065976e4074ae8481e69bd64a34-a5cb3fd4064a458a","baggage":"sentry-trace_id=2dfde065976e4074ae8481e69bd64a34,sentry-release=Unknown"},"correlation_id":"01M7A7T3VBGCY888589SVN5M8J","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:10.175Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-4f3ddc729b725845f483f2b2: done: 0.045917 sec","job_status":"done","queue_duration_s":2.042369,"scheduling_latency_s":2.042386,"redis_calls":10,"redis_duration_s":0.006459,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000328,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005681,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00045,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029415,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025496,"duration_s":0.045917,"completed_at":"2026-11-13T10:40:12.264Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005677,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1952gitlab # [ 403.611836] gitaly[905]: time="2026-11-13T10:40:12.201Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WNVD33SDQXCEHX35CC1G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:40:42.100" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:12.200" grpc.time_ms=0.583 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1953gitlab # [ 403.625406] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:40:12 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/5c12ba2ba5cde5b1dd1bfa095c2839632222f25c HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1411954gitlab # [ 403.627964] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:12.265Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"053b4f32a56b3f05c5713900","created_at":"2026-11-13T10:40:10.183Z","trace_propagation_headers":{"sentry-trace":"2dfde065976e4074ae8481e69bd64a34-a5cb3fd4064a458a","baggage":"sentry-trace_id=2dfde065976e4074ae8481e69bd64a34,sentry-release=Unknown"},"correlation_id":"01M7A7T3VBGCY888589SVN5M8J","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:10.187Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-053b4f32a56b3f05c5713900: start","job_status":"start","queue_duration_s":2.077669,"scheduling_latency_s":2.077687}1955gitlab # [ 403.645192] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:12.309Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"053b4f32a56b3f05c5713900","created_at":"2026-11-13T10:40:10.183Z","trace_propagation_headers":{"sentry-trace":"2dfde065976e4074ae8481e69bd64a34-a5cb3fd4064a458a","baggage":"sentry-trace_id=2dfde065976e4074ae8481e69bd64a34,sentry-release=Unknown"},"correlation_id":"01M7A7T3VBGCY888589SVN5M8J","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:10.187Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-053b4f32a56b3f05c5713900: done: 0.044218 sec","job_status":"done","queue_duration_s":2.077669,"scheduling_latency_s":2.077687,"redis_calls":10,"redis_duration_s":0.00531,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000507,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003781,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001022,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023924,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000503,"duration_s":0.044218,"completed_at":"2026-11-13T10:40:12.309Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009331,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1956gitlab # [ 403.691880] gitaly[905]: time="2026-11-13T10:40:12.269Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7WNVD33SDQXCEHX35CC1G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-13T10:40:22.070" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:40:12.260" grpc.time_ms=9.091 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1957gitlab # [ 403.706570] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:40:12 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/5c12ba2ba5cde5b1dd1bfa095c2839632222f25c HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1321958gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5'1959gitlab # [ 403.710271] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:12.311Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"898cb8beb6d44ac765abd8f0","created_at":"2026-11-13T10:40:10.188Z","trace_propagation_headers":{"sentry-trace":"2dfde065976e4074ae8481e69bd64a34-a5cb3fd4064a458a","baggage":"sentry-trace_id=2dfde065976e4074ae8481e69bd64a34,sentry-release=Unknown"},"correlation_id":"01M7A7T3VBGCY888589SVN5M8J","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:10.193Z","job_size_bytes":6,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-898cb8beb6d44ac765abd8f0: start","job_status":"start","queue_duration_s":2.118128,"scheduling_latency_s":2.11815}1960gitlab # [ 403.726418] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:12.342Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"898cb8beb6d44ac765abd8f0","created_at":"2026-11-13T10:40:10.188Z","trace_propagation_headers":{"sentry-trace":"2dfde065976e4074ae8481e69bd64a34-a5cb3fd4064a458a","baggage":"sentry-trace_id=2dfde065976e4074ae8481e69bd64a34,sentry-release=Unknown"},"correlation_id":"01M7A7T3VBGCY888589SVN5M8J","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:10.193Z","job_size_bytes":6,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-898cb8beb6d44ac765abd8f0: done: 0.031646 sec","job_status":"done","queue_duration_s":2.118128,"scheduling_latency_s":2.11815,"redis_calls":10,"redis_duration_s":0.002205,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00047,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001367,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000368,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019746,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002444,"duration_s":0.031646,"completed_at":"2026-11-13T10:40:12.342Z","load_balancing_strategy":"primary","db_duration_s":0.002764,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1961gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5', in 0.06 seconds)1962gitlab # [ 403.770773] gitaly[905]: time="2026-11-13T10:40:12.414Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WP26PFXCCXBTTKRBHC9Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:40:42.313" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:12.413" grpc.time_ms=0.644 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1963gitlab # [ 403.785406] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:12.344Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"0f118e6fff9f0a96dbab3d61","created_at":"2026-11-13T10:40:10.193Z","trace_propagation_headers":{"sentry-trace":"2dfde065976e4074ae8481e69bd64a34-a5cb3fd4064a458a","baggage":"sentry-trace_id=2dfde065976e4074ae8481e69bd64a34,sentry-release=Unknown"},"correlation_id":"01M7A7T3VBGCY888589SVN5M8J","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:10.197Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-0f118e6fff9f0a96dbab3d61: start","job_status":"start","queue_duration_s":2.146803,"scheduling_latency_s":2.146825}1964gitlab # [ 403.801006] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:12.394Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"0f118e6fff9f0a96dbab3d61","created_at":"2026-11-13T10:40:10.193Z","trace_propagation_headers":{"sentry-trace":"2dfde065976e4074ae8481e69bd64a34-a5cb3fd4064a458a","baggage":"sentry-trace_id=2dfde065976e4074ae8481e69bd64a34,sentry-release=Unknown"},"correlation_id":"01M7A7T3VBGCY888589SVN5M8J","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:10.197Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-0f118e6fff9f0a96dbab3d61: done: 0.049978 sec","job_status":"done","queue_duration_s":2.146803,"scheduling_latency_s":2.146825,"redis_calls":10,"redis_duration_s":0.003849,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000416,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002789,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000644,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030279,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00054536,"duration_s":0.049978,"completed_at":"2026-11-13T10:40:12.394Z","load_balancing_strategy":"primary","db_duration_s":0.010817,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1965gitlab # [ 403.845293] nixbot[3443]: {"ts": "2026-11-13T10:40:12.719Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:47672 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5 HTTP/1.1\" 200"}1966gitlab # [ 403.848119] gitaly[905]: time="2026-11-13T10:40:12.456Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7WP26PFXCCXBTTKRBHC9Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-13T10:40:22.257" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:40:12.448" grpc.time_ms=7.915 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1967gitlab # [ 403.861876] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:12.397Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"f82ec9d5bcba028aabd20908","created_at":"2026-11-13T10:40:10.203Z","trace_propagation_headers":{"sentry-trace":"2dfde065976e4074ae8481e69bd64a34-a5cb3fd4064a458a","baggage":"sentry-trace_id=2dfde065976e4074ae8481e69bd64a34,sentry-release=Unknown"},"correlation_id":"01M7A7T3VBGCY888589SVN5M8J","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:10.208Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-f82ec9d5bcba028aabd20908: start","job_status":"start","queue_duration_s":2.189571,"scheduling_latency_s":2.189596}1968gitlab # [ 403.879331] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:12.474Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"f82ec9d5bcba028aabd20908","created_at":"2026-11-13T10:40:10.203Z","trace_propagation_headers":{"sentry-trace":"2dfde065976e4074ae8481e69bd64a34-a5cb3fd4064a458a","baggage":"sentry-trace_id=2dfde065976e4074ae8481e69bd64a34,sentry-release=Unknown"},"correlation_id":"01M7A7T3VBGCY888589SVN5M8J","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:10.208Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-f82ec9d5bcba028aabd20908: done: 0.0768 sec","job_status":"done","queue_duration_s":2.189571,"scheduling_latency_s":2.189596,"redis_calls":14,"redis_duration_s":0.005711,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000415,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002531,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002765,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.052044,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00029916,"duration_s":0.0768,"completed_at":"2026-11-13T10:40:12.474Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014089,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1969gitlab # [ 403.924679] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:40:12 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 671970gitlab # [ 403.926899] gitlab-workhorse[818]: time="2026-11-13T10:40:12Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671971gitlab # [ 403.928808] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:40:12 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 761972gitlab # [ 403.930763] gitaly[905]: time="2026-11-13T10:40:12.554Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WP6VJM8EVXYS8T3R4TED grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:40:42.453" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:12.554" grpc.time_ms=0.509 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1973gitlab # [ 403.943927] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:12.609Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"66b72bd3c1f984c6b2b1a73b","created_at":"2026-11-13T10:40:10.345Z","trace_propagation_headers":{"sentry-trace":"597dbc31f9ab46c4a245d75aa6dcd926-bf0636f0bb8f4dd1","baggage":"sentry-trace_id=597dbc31f9ab46c4a245d75aa6dcd926,sentry-release=Unknown"},"correlation_id":"01M7A7WFGXP3SWPPDZ22MN1RGB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"8ad0852bf785891bc8dc7029","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:10.348Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-66b72bd3c1f984c6b2b1a73b: start","job_status":"start","queue_duration_s":2.260628,"scheduling_latency_s":2.260658}1974gitlab # [ 403.961614] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:12.667Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"66b72bd3c1f984c6b2b1a73b","created_at":"2026-11-13T10:40:10.345Z","trace_propagation_headers":{"sentry-trace":"597dbc31f9ab46c4a245d75aa6dcd926-bf0636f0bb8f4dd1","baggage":"sentry-trace_id=597dbc31f9ab46c4a245d75aa6dcd926,sentry-release=Unknown"},"correlation_id":"01M7A7WFGXP3SWPPDZ22MN1RGB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"8ad0852bf785891bc8dc7029","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:10.348Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-66b72bd3c1f984c6b2b1a73b: done: 0.058072 sec","job_status":"done","queue_duration_s":2.260628,"scheduling_latency_s":2.260658,"gitaly_calls":1,"gitaly_duration_s":0.012529,"redis_calls":12,"redis_duration_s":0.002509,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000922,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000595,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000992,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.009,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.009,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034373,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0347777,"duration_s":0.058072,"completed_at":"2026-11-13T10:40:12.667Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009728,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1975gitlab # [ 404.008334] gitaly[905]: time="2026-11-13T10:40:12.595Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M7A7WP6VJM8EVXYS8T3R4TED grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-13T10:40:22.055" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:40:12.585" grpc.time_ms=9.663 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1976gitlab # [ 404.022330] gitaly[905]: time="2026-11-13T10:40:12.644Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WFGXP3SWPPDZ22MN1RGB diskcache=57997261-9dc3-415b-bda8-3d04d7b8fe89 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-11-13T16:40:12.633" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:12.634" pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1977gitlab # [ 404.033125] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:40:12 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 651978gitlab # [ 404.035272] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:12.669Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"07e2ed36f9ba069d32132a6e","created_at":"2026-11-13T10:40:10.471Z","trace_propagation_headers":{"sentry-trace":"597dbc31f9ab46c4a245d75aa6dcd926-bf0636f0bb8f4dd1","baggage":"sentry-trace_id=597dbc31f9ab46c4a245d75aa6dcd926,sentry-release=Unknown"},"correlation_id":"01M7A7WFGXP3SWPPDZ22MN1RGB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:50c668b13037faeb9c5b5bca3a55360381c3847fa2fd09f41b715800cf166759","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:10.473Z","job_size_bytes":467,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-07e2ed36f9ba069d32132a6e: start","job_status":"start","queue_duration_s":2.1954,"scheduling_latency_s":2.195424}1979gitlab # [ 404.053225] gitaly[905]: time="2026-11-13T10:40:12.644Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=289 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WFGXP3SWPPDZ22MN1RGB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-11-13T16:40:12.633" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:12.634" grpc.time_ms=9.887 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1980gitlab # [ 404.066847] gitlab-workhorse[818]: time="2026-11-13T10:40:12Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671981gitlab # [ 404.069701] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:40:12 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 641982gitlab # [ 404.071640] gitaly[905]: time="2026-11-13T10:40:12.702Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WFGXP3SWPPDZ22MN1RGB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:40:42.501" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:12.701" grpc.time_ms=1.128 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1983gitlab # [ 404.084725] gitaly[905]: time="2026-11-13T10:40:12.719Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=8 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WFGXP3SWPPDZ22MN1RGB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-11-13T10:40:22.007" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:40:12.707" grpc.time_ms=11.176 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1984gitlab # [ 404.098124] gitaly[905]: time="2026-11-13T10:40:12.746Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M7A7WPC6DDQ01H270MFK33PQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:40:12.738" grpc.time_ms=7.528 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1985gitlab # [ 404.111565] gitaly[905]: time="2026-11-13T10:40:12.755Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7WFGXP3SWPPDZ22MN1RGB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-13T10:40:42.054" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:12.754" grpc.time_ms=0.8 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1986gitlab # [ 404.124893] gitaly[905]: time="2026-11-13T10:40:12.761Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=38 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7WFGXP3SWPPDZ22MN1RGB error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-11-13T10:40:42.058" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:12.759" grpc.time_ms=2.093 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1987gitlab # [ 404.141518] gitaly[905]: time="2026-11-13T10:40:12.794Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7WFGXP3SWPPDZ22MN1RGB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-13T10:40:22.086" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:40:12.787" grpc.time_ms=7.412 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1988gitlab # [ 404.155351] gitaly[905]: time="2026-11-13T10:40:12.823Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WPECWZBNJYT6999HQWXN grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:40:12.813" pid=905 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1989gitlab # [ 404.165868] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:40:13 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 571990gitlab # [ 404.167973] gitaly[905]: time="2026-11-13T10:40:12.823Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WPECWZBNJYT6999HQWXN grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:40:12.813" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1991gitlab # [ 404.181977] gitaly[905]: time="2026-11-13T10:40:12.825Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WPECWZBNJYT6999HQWXN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:40:12.813" grpc.time_ms=10.159 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1992gitlab # [ 404.196793] gitaly[905]: time="2026-11-13T10:40:12.924Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=3 command.system_time_ms=0 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M7A7WPHV3MXVEN9PK6K5062R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=206 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:40:12.912" grpc.time_ms=12.549 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1993gitlab # [ 404.210246] gitaly[905]: time="2026-11-13T10:40:12.991Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WPKY1CR3DFTHFJ6123XE grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:40:12.982" pid=905 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc1994gitlab # [ 404.220719] gitaly[905]: time="2026-11-13T10:40:12.991Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WPKY1CR3DFTHFJ6123XE grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:40:12.982" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1995gitlab # [ 404.234468] gitaly[905]: time="2026-11-13T10:40:12.991Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WPKY1CR3DFTHFJ6123XE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:40:12.982" grpc.time_ms=9.282 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1996gitlab # [ 404.250083] gitaly[905]: time="2026-11-13T10:40:13.086Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=3 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M7A7WPQ5NCJMH392TQEK0HZR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:40:13.077" grpc.time_ms=9.147 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1997gitlab # [ 404.263658] gitaly[905]: time="2026-11-13T10:40:13.183Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WPS053FNP6BYQSW2D5D0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:40:13.162" pid=905 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc1998gitlab # [ 404.275022] gitlab-workhorse[818]: time="2026-11-13T10:40:13Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2161999gitlab # [ 404.276975] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:40:13 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 962000gitlab # [ 404.278888] gitaly[905]: time="2026-11-13T10:40:13.183Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1102 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=1 command.system_time_ms=12 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WPS053FNP6BYQSW2D5D0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:40:13.162" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc2001gitlab # [ 404.293181] gitaly[905]: time="2026-11-13T10:40:13.184Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1102 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=1 command.system_time_ms=12 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WPS053FNP6BYQSW2D5D0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:40:13.162" grpc.time_ms=21.55 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc2002gitlab # [ 404.370087] gitaly[905]: time="2026-11-13T10:40:13.319Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=883 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M7A7WFGXP3SWPPDZ22MN1RGB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-11-13T10:40:43.004" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:13.304" grpc.time_ms=14.312 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2003gitlab # [ 404.394880] gitaly[905]: time="2026-11-13T10:40:13.333Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7WFGXP3SWPPDZ22MN1RGB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-11-13T10:40:43.022" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:13.322" grpc.time_ms=11.167 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2004gitlab # [ 404.415844] sidekiq-cluster[3321]: {"severity":"WARN","time":"2026-11-13T10:40:13.347Z","class":"CreatePipelineWorker","correlation_id":"01M7A7WFGXP3SWPPDZ22MN1RGB","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"5c12ba2ba5cde5b1dd1bfa095c2839632222f25c","after":"6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5","ref":"refs/heads/master"},"retry":0}2005gitlab # [ 404.426021] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:13.352Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"07e2ed36f9ba069d32132a6e","created_at":"2026-11-13T10:40:10.471Z","trace_propagation_headers":{"sentry-trace":"597dbc31f9ab46c4a245d75aa6dcd926-bf0636f0bb8f4dd1","baggage":"sentry-trace_id=597dbc31f9ab46c4a245d75aa6dcd926,sentry-release=Unknown"},"correlation_id":"01M7A7WFGXP3SWPPDZ22MN1RGB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:50c668b13037faeb9c5b5bca3a55360381c3847fa2fd09f41b715800cf166759","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:10.473Z","job_size_bytes":467,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-07e2ed36f9ba069d32132a6e: done: 0.683489 sec","job_status":"done","queue_duration_s":2.1954,"scheduling_latency_s":2.195424,"gitaly_calls":7,"gitaly_duration_s":0.066993,"redis_calls":33,"redis_duration_s":0.005852,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000484,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00165,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002827,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000891,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.597372,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.683489,"completed_at":"2026-11-13T10:40:13.352Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.018365,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2006gitlab # [ 404.483904] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:13.354Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"3b6d5541-a6f2-4d4a-b734-564d0336e9c1\"}"],"class":"WebHookWorker","jid":"f225b42cf7dfc27cb648f48c","created_at":"2026-11-13T10:40:10.491Z","trace_propagation_headers":{"sentry-trace":"597dbc31f9ab46c4a245d75aa6dcd926-bf0636f0bb8f4dd1","baggage":"sentry-trace_id=597dbc31f9ab46c4a245d75aa6dcd926,sentry-release=Unknown"},"correlation_id":"01M7A7WFGXP3SWPPDZ22MN1RGB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:9fa57e8f78fdffcf8fe670fe5bf6635a66564ff483f86d729fa5aba72ed19141","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:10.494Z","job_size_bytes":1749,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-f225b42cf7dfc27cb648f48c: start","job_status":"start","queue_duration_s":2.860184,"scheduling_latency_s":2.860211}2007gitlab # [ 404.503646] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:13.395Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"3b6d5541-a6f2-4d4a-b734-564d0336e9c1\"}"],"class":"WebHookWorker","jid":"f225b42cf7dfc27cb648f48c","created_at":"2026-11-13T10:40:10.491Z","trace_propagation_headers":{"sentry-trace":"597dbc31f9ab46c4a245d75aa6dcd926-bf0636f0bb8f4dd1","baggage":"sentry-trace_id=597dbc31f9ab46c4a245d75aa6dcd926,sentry-release=Unknown"},"correlation_id":"01M7A7WFGXP3SWPPDZ22MN1RGB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:9fa57e8f78fdffcf8fe670fe5bf6635a66564ff483f86d729fa5aba72ed19141","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:10.494Z","job_size_bytes":1749,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-f225b42cf7dfc27cb648f48c: done: 0.041134 sec","job_status":"done","queue_duration_s":2.860184,"scheduling_latency_s":2.860211,"redis_calls":14,"redis_duration_s":0.002624,"redis_read_bytes":25,"redis_write_bytes":4685,"redis_queues_calls":2,"redis_queues_duration_s":0.000304,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3729,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001535,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000785,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.007226360000004206,"cpu_s":0.025714,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.041134,"completed_at":"2026-11-13T10:40:13.395Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002265,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2008gitlab # [ 404.552674] nixbot[3443]: {"ts": "2026-11-13T10:40:13.382Z", "level": "info", "logger": "nixbot.webhooks", "msg": "webhook accepted", "delivery": "79643361-eb14-44e9-81b2-2dd674ca209d", "event": "ChangeRequest", "forge": "gitlab", "repo_id": "1", "branch": "master", "sha": "6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5", "pr": null}2009gitlab # [ 404.556375] nixbot[3443]: {"ts": "2026-11-13T10:40:13.385Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:47676 - \"POST /webhooks/gitlab HTTP/1.1\" 202"}2010gitlab # [ 404.558425] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:40:13 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 732011gitlab # [ 404.560600] gitaly[905]: time="2026-11-13T10:40:13.484Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M7A7WQ33KJ7BGNHKNFY8DPE4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:40:13.473" grpc.time_ms=10.736 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc2012gitlab # [ 404.574131] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:13.397Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"78f7c7dee86c4d312e893ac9","created_at":"2026-11-13T10:40:10.617Z","trace_propagation_headers":{"sentry-trace":"1fa6eccd68f44a29905cb41224adbb6d-dbd038f478cf4727","baggage":"sentry-trace_id=1fa6eccd68f44a29905cb41224adbb6d,sentry-release=Unknown"},"correlation_id":"01M7A7SY1QCQ856G1GYR454CPX","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:10.623Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-78f7c7dee86c4d312e893ac9: start","job_status":"start","queue_duration_s":2.773432,"scheduling_latency_s":2.77345}2013gitlab # [ 404.590624] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:13.447Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"78f7c7dee86c4d312e893ac9","created_at":"2026-11-13T10:40:10.617Z","trace_propagation_headers":{"sentry-trace":"1fa6eccd68f44a29905cb41224adbb6d-dbd038f478cf4727","baggage":"sentry-trace_id=1fa6eccd68f44a29905cb41224adbb6d,sentry-release=Unknown"},"correlation_id":"01M7A7SY1QCQ856G1GYR454CPX","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:10.623Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-78f7c7dee86c4d312e893ac9: done: 0.050435 sec","job_status":"done","queue_duration_s":2.773432,"scheduling_latency_s":2.77345,"redis_calls":19,"redis_duration_s":0.00348,"redis_read_bytes":21,"redis_write_bytes":4291,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000828,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000464,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2262,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001538,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00065,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036967,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.007878679999999999,"duration_s":0.050435,"completed_at":"2026-11-13T10:40:13.447Z","load_balancing_strategy":"primary","db_duration_s":0.007323,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2014gitlab # [ 404.637553] gitaly[905]: time="2026-11-13T10:40:13.561Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WQ5EKD4FQ0ERR5C6DTJS grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:40:13.550" pid=905 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2015gitlab # [ 404.648100] gitlab-workhorse[818]: time="2026-11-13T10:40:13Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672016gitlab # [ 404.649967] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:40:13 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 752017gitlab # [ 404.651850] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:13.449Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","24e8e3a419bcfa9d03a9208e"],"class":"WebHooks::LogExecutionWorker","jid":"8315f48f3356a269c8b08d19","created_at":"2026-11-13T10:40:11.828Z","trace_propagation_headers":{"sentry-trace":"1fa6eccd68f44a29905cb41224adbb6d-d046bcb5a68449db","baggage":"sentry-trace_id=1fa6eccd68f44a29905cb41224adbb6d,sentry-release=Unknown"},"correlation_id":"01M7A7SY1QCQ856G1GYR454CPX","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:11b406ac650eef3982df2a45ed7cf61a539559265fea0709ba26cd70c0e6d721","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:11.837Z","job_size_bytes":2592,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-8315f48f3356a269c8b08d19: start","job_status":"start","queue_duration_s":1.611326,"scheduling_latency_s":1.611346}2018gitlab # [ 404.669305] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:13.494Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","24e8e3a419bcfa9d03a9208e"],"class":"WebHooks::LogExecutionWorker","jid":"8315f48f3356a269c8b08d19","created_at":"2026-11-13T10:40:11.828Z","trace_propagation_headers":{"sentry-trace":"1fa6eccd68f44a29905cb41224adbb6d-d046bcb5a68449db","baggage":"sentry-trace_id=1fa6eccd68f44a29905cb41224adbb6d,sentry-release=Unknown"},"correlation_id":"01M7A7SY1QCQ856G1GYR454CPX","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:11b406ac650eef3982df2a45ed7cf61a539559265fea0709ba26cd70c0e6d721","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:11.837Z","job_size_bytes":2592,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-8315f48f3356a269c8b08d19: done: 0.045244 sec","job_status":"done","queue_duration_s":1.611326,"scheduling_latency_s":1.611346,"redis_calls":10,"redis_duration_s":0.0017339999999999999,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000348,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001022,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000364,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031257,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025852,"duration_s":0.045244,"completed_at":"2026-11-13T10:40:13.494Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008126,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2019gitlab # [ 404.717082] gitaly[905]: time="2026-11-13T10:40:13.561Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WQ5EKD4FQ0ERR5C6DTJS grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:40:13.550" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc2020gitlab # [ 404.731354] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:13.495Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"2398065df1c7164489fcdfb6","created_at":"2026-11-13T10:40:12.327Z","trace_propagation_headers":{"sentry-trace":"e26f107a95474185bf2c205248c944b5-d580c2dbc392496f","baggage":"sentry-trace_id=e26f107a95474185bf2c205248c944b5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7WNVD33SDQXCEHX35CC1G","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:12.332Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-2398065df1c7164489fcdfb6: start","job_status":"start","queue_duration_s":1.163299,"scheduling_latency_s":1.163316}2021gitlab # [ 404.748541] gitaly[905]: time="2026-11-13T10:40:13.561Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WQ5EKD4FQ0ERR5C6DTJS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:40:13.550" grpc.time_ms=11.23 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc2022gitlab # [ 404.763462] gitaly[905]: time="2026-11-13T10:40:13.605Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7WNVD33SDQXCEHX35CC1G grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3897 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:40:13.605" grpc.time_ms=0.713 pid=905 span.kind=server system=grpc2023gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5'2024gitlab # [ 404.773725] gitaly[905]: time="2026-11-13T10:40:13.630Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7WNVD33SDQXCEHX35CC1G grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3895 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:40:13.630" grpc.time_ms=0.492 pid=905 span.kind=server system=grpc2025gitlab # [ 404.783347] gitaly[905]: time="2026-11-13T10:40:13.655Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7WNVD33SDQXCEHX35CC1G grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3897 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:40:13.653" grpc.time_ms=1.886 pid=905 span.kind=server system=grpc2026gitlab # [ 404.792766] gitaly[905]: time="2026-11-13T10:40:13.661Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WNVD33SDQXCEHX35CC1G diskcache=40adfb7f-49ef-472c-8f06-d9e5f2fbcc57 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-13T10:40:43.274" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:40:13.575" pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2027gitlab # [ 404.802770] gitaly[905]: time="2026-11-13T10:40:13.662Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=82 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7319 command.oublock=0 command.real_time_ms=83 command.spawn_token_fork_ms=0 command.system_time_ms=58 command.user_time_ms=24 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WNVD33SDQXCEHX35CC1G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-13T10:40:43.274" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:40:13.575" grpc.time_ms=86.972 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2028gitlab # [ 404.816594] gitaly[905]: time="2026-11-13T10:40:13.664Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WQ8XNXM66JRPDK5TD9N4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:40:43.563" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:13.664" grpc.time_ms=0.599 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2029gitlab # [ 404.830217] gitaly[905]: time="2026-11-13T10:40:13.684Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=591 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7WQ8XNXM66JRPDK5TD9N4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-11-13T10:40:43.575" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:40:13.675" grpc.time_ms=8.364 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2030gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5', in 0.07 seconds)2031(finished: subtest: push delivers the webhook, in 86.01 seconds)2032subtest: eval and build post commit statuses2033gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5/statuses2034gitlab # [ 404.845441] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:13.769Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"2398065df1c7164489fcdfb6","created_at":"2026-11-13T10:40:12.327Z","trace_propagation_headers":{"sentry-trace":"e26f107a95474185bf2c205248c944b5-d580c2dbc392496f","baggage":"sentry-trace_id=e26f107a95474185bf2c205248c944b5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7WNVD33SDQXCEHX35CC1G","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:12.332Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":3321,"job_status":"rescheduled","message":"PipelineProcessWorker JID-2398065df1c7164489fcdfb6: rescheduled"}2035gitlab # [ 404.861634] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:13.776Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"2398065df1c7164489fcdfb6","created_at":"2026-11-13T10:40:12.327Z","trace_propagation_headers":{"sentry-trace":"e26f107a95474185bf2c205248c944b5-d580c2dbc392496f","baggage":"sentry-trace_id=e26f107a95474185bf2c205248c944b5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7WNVD33SDQXCEHX35CC1G","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:12.332Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-2398065df1c7164489fcdfb6: done: 0.281217 sec","job_status":"done","queue_duration_s":1.163299,"scheduling_latency_s":1.163316,"gitaly_calls":3,"gitaly_duration_s":0.098595,"redis_calls":66,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.011447,"redis_read_bytes":148,"redis_write_bytes":18013,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000365,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000787,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.002654,"redis_queues_read_bytes":26,"redis_queues_write_bytes":12312,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.004256,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000185,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000376,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.002459,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.045,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.007,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.007,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.028,"db_ci_duration_s":0.017,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.135161,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.23969828,"duration_s":0.281217,"completed_at":"2026-11-13T10:40:13.776Z","load_balancing_strategy":"primary","db_duration_s":0.045809,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2036gitlab # [ 404.916261] gitaly[905]: time="2026-11-13T10:40:13.700Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=3 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7WQ8XNXM66JRPDK5TD9N4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-13T10:40:23.031" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:40:13.692" grpc.time_ms=8.494 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2037gitlab # [ 404.930496] nixbot[3443]: {"ts": "2026-11-13T10:40:13.780Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:47690 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5 HTTP/1.1\" 200"}2038gitlab # [ 404.933501] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:13.778Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","f225b42cf7dfc27cb648f48c"],"class":"WebHooks::LogExecutionWorker","jid":"e59cf107c99a0797501506b8","created_at":"2026-11-13T10:40:13.386Z","trace_propagation_headers":{"sentry-trace":"597dbc31f9ab46c4a245d75aa6dcd926-f230ad5ffebc4e36","baggage":"sentry-trace_id=597dbc31f9ab46c4a245d75aa6dcd926,sentry-release=Unknown"},"correlation_id":"01M7A7WFGXP3SWPPDZ22MN1RGB","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fe53c6ef737aea6939b2774eb487b1fe7e98da4e3b77aed278267f56fc6370cb","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:13.389Z","job_size_bytes":2592,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-e59cf107c99a0797501506b8: start","job_status":"start","queue_duration_s":0.389233,"scheduling_latency_s":0.389258}2039gitlab # [ 404.951180] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:13.805Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","f225b42cf7dfc27cb648f48c"],"class":"WebHooks::LogExecutionWorker","jid":"e59cf107c99a0797501506b8","created_at":"2026-11-13T10:40:13.386Z","trace_propagation_headers":{"sentry-trace":"597dbc31f9ab46c4a245d75aa6dcd926-f230ad5ffebc4e36","baggage":"sentry-trace_id=597dbc31f9ab46c4a245d75aa6dcd926,sentry-release=Unknown"},"correlation_id":"01M7A7WFGXP3SWPPDZ22MN1RGB","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fe53c6ef737aea6939b2774eb487b1fe7e98da4e3b77aed278267f56fc6370cb","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:13.389Z","job_size_bytes":2592,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-e59cf107c99a0797501506b8: done: 0.027177 sec","job_status":"done","queue_duration_s":0.389233,"scheduling_latency_s":0.389258,"redis_calls":7,"redis_duration_s":0.0017980000000000001,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000936,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000862,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01961,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027177,"completed_at":"2026-11-13T10:40:13.805Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002829,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2040gitlab # [ 404.996077] gitaly[905]: time="2026-11-13T10:40:13.719Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7WNVD33SDQXCEHX35CC1G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-13T10:40:43.018" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:13.718" grpc.time_ms=1.136 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2041gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5/statuses, in 0.17 seconds)2042gitlab # [ 405.009606] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:40:13 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 3472043gitlab # [ 405.012168] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:40:13 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 962044gitlab # [ 405.015032] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:13.806Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"0c74b7af9da3609f26ee4bb4","created_at":"2026-11-13T10:40:13.433Z","trace_propagation_headers":{"sentry-trace":"1fa6eccd68f44a29905cb41224adbb6d-692e81711a7d4541","baggage":"sentry-trace_id=1fa6eccd68f44a29905cb41224adbb6d,sentry-release=Unknown"},"correlation_id":"01M7A7SY1QCQ856G1GYR454CPX","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:13.438Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-0c74b7af9da3609f26ee4bb4: start","job_status":"start","queue_duration_s":0.367907,"scheduling_latency_s":0.367926}2045gitlab # [ 405.032508] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:13.914Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"0c74b7af9da3609f26ee4bb4","created_at":"2026-11-13T10:40:13.433Z","trace_propagation_headers":{"sentry-trace":"1fa6eccd68f44a29905cb41224adbb6d-692e81711a7d4541","baggage":"sentry-trace_id=1fa6eccd68f44a29905cb41224adbb6d,sentry-release=Unknown"},"correlation_id":"01M7A7SY1QCQ856G1GYR454CPX","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:13.438Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-0c74b7af9da3609f26ee4bb4: done: 0.10816 sec","job_status":"done","queue_duration_s":0.367907,"scheduling_latency_s":0.367926,"redis_calls":11,"redis_duration_s":0.002035,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000318,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001374,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000343,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.01,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.075731,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026948,"duration_s":0.10816,"completed_at":"2026-11-13T10:40:13.914Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.028165,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2046gitlab # [ 405.078580] gitaly[905]: time="2026-11-13T10:40:13.736Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WQ8XNXM66JRPDK5TD9N4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-11-13T10:40:23.075" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:40:13.735" grpc.time_ms=0.273 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2047gitlab # [ 405.089880] nixbot[3443]: {"ts": "2026-11-13T10:40:13.958Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-6f5cdc2f13a6-6"}2048gitlab # [ 405.092266] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:13.916Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"88685f2fe34973fc73b2b3b2","created_at":"2026-11-13T10:40:13.564Z","trace_propagation_headers":{"sentry-trace":"e26f107a95474185bf2c205248c944b5-6bd9f592984c4b2c","baggage":"sentry-trace_id=e26f107a95474185bf2c205248c944b5,sentry-release=Unknown"},"correlation_id":"01M7A7WNVD33SDQXCEHX35CC1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:13.567Z","job_size_bytes":23,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-88685f2fe34973fc73b2b3b2: start","job_status":"start","queue_duration_s":0.348903,"scheduling_latency_s":0.348925}2049gitlab # [ 405.110106] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:13.966Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"88685f2fe34973fc73b2b3b2","created_at":"2026-11-13T10:40:13.564Z","trace_propagation_headers":{"sentry-trace":"e26f107a95474185bf2c205248c944b5-6bd9f592984c4b2c","baggage":"sentry-trace_id=e26f107a95474185bf2c205248c944b5,sentry-release=Unknown"},"correlation_id":"01M7A7WNVD33SDQXCEHX35CC1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:13.567Z","job_size_bytes":23,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-88685f2fe34973fc73b2b3b2: done: 0.05053 sec","job_status":"done","queue_duration_s":0.348903,"scheduling_latency_s":0.348925,"redis_calls":7,"redis_duration_s":0.0014680000000000001,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001002,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000466,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034529,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.05053,"completed_at":"2026-11-13T10:40:13.966Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011265,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2050gitlab # [ 405.153931] gitaly[905]: time="2026-11-13T10:40:13.749Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=570 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WQ8XNXM66JRPDK5TD9N4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-11-13T10:40:23.080" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:40:13.741" grpc.time_ms=7.516 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2051gitlab # [ 405.167547] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:13.968Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"3bc1ca8a3412ca43ee653a08","created_at":"2026-11-13T10:40:13.667Z","trace_propagation_headers":{"sentry-trace":"e26f107a95474185bf2c205248c944b5-6bd9f592984c4b2c","baggage":"sentry-trace_id=e26f107a95474185bf2c205248c944b5,sentry-release=Unknown"},"correlation_id":"01M7A7WNVD33SDQXCEHX35CC1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:13.669Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-3bc1ca8a3412ca43ee653a08: start","job_status":"start","queue_duration_s":0.298561,"scheduling_latency_s":0.29858}2052gitlab # [ 405.184632] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:14.001Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"3bc1ca8a3412ca43ee653a08","created_at":"2026-11-13T10:40:13.667Z","trace_propagation_headers":{"sentry-trace":"e26f107a95474185bf2c205248c944b5-6bd9f592984c4b2c","baggage":"sentry-trace_id=e26f107a95474185bf2c205248c944b5,sentry-release=Unknown"},"correlation_id":"01M7A7WNVD33SDQXCEHX35CC1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:13.669Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-3bc1ca8a3412ca43ee653a08: done: 0.033198 sec","job_status":"done","queue_duration_s":0.298561,"scheduling_latency_s":0.29858,"redis_calls":7,"redis_duration_s":0.001206,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000904,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000302,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022637,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033198,"completed_at":"2026-11-13T10:40:14.001Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00592,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2053gitlab # [ 405.228742] nixbot[3443]: {"ts": "2026-11-13T10:40:14.080Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-6f5cdc2f13a6-6", "argv0": "bwrap"}2054gitlab # [ 405.231538] gitaly[905]: time="2026-11-13T10:40:13.757Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WNVD33SDQXCEHX35CC1G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:40:43.056" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:13.756" grpc.time_ms=0.686 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2055gitlab # [ 405.244633] gitaly[905]: time="2026-11-13T10:40:13.786Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7WQ8XNXM66JRPDK5TD9N4 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4153 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:40:13.785" grpc.time_ms=1.021 pid=905 span.kind=server system=grpc2056gitlab # [ 405.254474] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:14.003Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"20a67a4f330dc4e2552d4337","created_at":"2026-11-13T10:40:13.681Z","trace_propagation_headers":{"sentry-trace":"e26f107a95474185bf2c205248c944b5-6bd9f592984c4b2c","baggage":"sentry-trace_id=e26f107a95474185bf2c205248c944b5,sentry-release=Unknown"},"correlation_id":"01M7A7WNVD33SDQXCEHX35CC1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:13.683Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-20a67a4f330dc4e2552d4337: start","job_status":"start","queue_duration_s":0.319549,"scheduling_latency_s":0.319574}2057gitlab # [ 405.270723] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:14.037Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"20a67a4f330dc4e2552d4337","created_at":"2026-11-13T10:40:13.681Z","trace_propagation_headers":{"sentry-trace":"e26f107a95474185bf2c205248c944b5-6bd9f592984c4b2c","baggage":"sentry-trace_id=e26f107a95474185bf2c205248c944b5,sentry-release=Unknown"},"correlation_id":"01M7A7WNVD33SDQXCEHX35CC1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:13.683Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-20a67a4f330dc4e2552d4337: done: 0.034803 sec","job_status":"done","queue_duration_s":0.319549,"scheduling_latency_s":0.319574,"redis_calls":7,"redis_duration_s":0.0015919999999999999,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001102,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00049,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02527,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034803,"completed_at":"2026-11-13T10:40:14.037Z","load_balancing_strategy":"primary","db_duration_s":0.005151,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2058gitlab # [ 405.314867] gitaly[905]: time="2026-11-13T10:40:13.809Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7WQ8XNXM66JRPDK5TD9N4 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:40:13.808" grpc.time_ms=0.381 pid=905 span.kind=server system=grpc2059gitlab # [ 405.325012] gitaly[905]: time="2026-11-13T10:40:13.815Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WQ8XNXM66JRPDK5TD9N4 diskcache=c6e2de17-5a1b-4147-9304-28f711f110df grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-13T10:40:23.091" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:40:13.751" pid=905 ref=refs/keep-around/6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5 remote_ip="2001:db8:1::1" revision=6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5 span.kind=server system=grpc username=root2060gitlab # [ 405.336550] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:14.039Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"5dd08721a68d25ec378e4c46","created_at":"2026-11-13T10:40:13.691Z","trace_propagation_headers":{"sentry-trace":"e26f107a95474185bf2c205248c944b5-6bd9f592984c4b2c","baggage":"sentry-trace_id=e26f107a95474185bf2c205248c944b5,sentry-release=Unknown"},"correlation_id":"01M7A7WNVD33SDQXCEHX35CC1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:13.694Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-5dd08721a68d25ec378e4c46: start","job_status":"start","queue_duration_s":0.345543,"scheduling_latency_s":0.345576}2061gitlab # [ 405.352447] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:14.075Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"5dd08721a68d25ec378e4c46","created_at":"2026-11-13T10:40:13.691Z","trace_propagation_headers":{"sentry-trace":"e26f107a95474185bf2c205248c944b5-6bd9f592984c4b2c","baggage":"sentry-trace_id=e26f107a95474185bf2c205248c944b5,sentry-release=Unknown"},"correlation_id":"01M7A7WNVD33SDQXCEHX35CC1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:13.694Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-5dd08721a68d25ec378e4c46: done: 0.035995 sec","job_status":"done","queue_duration_s":0.345543,"scheduling_latency_s":0.345576,"redis_calls":10,"redis_duration_s":0.002271,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001192,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000468,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000611,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024818,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.035995,"completed_at":"2026-11-13T10:40:14.075Z","load_balancing_strategy":"primary","db_duration_s":0.005479,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2062gitlab # [ 405.396918] gitaly[905]: time="2026-11-13T10:40:13.816Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=53 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5400 command.oublock=8 command.real_time_ms=58 command.spawn_token_fork_ms=0 command.system_time_ms=20 command.user_time_ms=33 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WQ8XNXM66JRPDK5TD9N4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-13T10:40:23.091" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:40:13.751" grpc.time_ms=64.586 pid=905 ref=refs/keep-around/6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5 remote_ip="2001:db8:1::1" revision=6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5 span.kind=server system=grpc username=root2063gitlab # [ 405.412610] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:14.077Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"6428e51fccacc130bace2471","created_at":"2026-11-13T10:40:13.694Z","trace_propagation_headers":{"sentry-trace":"e26f107a95474185bf2c205248c944b5-6bd9f592984c4b2c","baggage":"sentry-trace_id=e26f107a95474185bf2c205248c944b5,sentry-release=Unknown"},"correlation_id":"01M7A7WNVD33SDQXCEHX35CC1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:13.696Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-6428e51fccacc130bace2471: start","job_status":"start","queue_duration_s":0.380769,"scheduling_latency_s":0.380787}2064gitlab # [ 405.429597] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:14.111Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"6428e51fccacc130bace2471","created_at":"2026-11-13T10:40:13.694Z","trace_propagation_headers":{"sentry-trace":"e26f107a95474185bf2c205248c944b5-6bd9f592984c4b2c","baggage":"sentry-trace_id=e26f107a95474185bf2c205248c944b5,sentry-release=Unknown"},"correlation_id":"01M7A7WNVD33SDQXCEHX35CC1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:13.696Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-6428e51fccacc130bace2471: done: 0.034222 sec","job_status":"done","queue_duration_s":0.380769,"scheduling_latency_s":0.380787,"redis_calls":7,"redis_duration_s":0.001438,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000982,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000456,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019202,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034222,"completed_at":"2026-11-13T10:40:14.111Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004732,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2065gitlab # [ 405.473474] gitaly[905]: time="2026-11-13T10:40:13.820Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WQ8XNXM66JRPDK5TD9N4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:40:43.019" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:13.819" grpc.time_ms=0.391 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2066gitlab # [ 405.486926] nix-daemon[3607]: accepted connection from pid 4683, user nixbot2067gitlab # [ 405.487931] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:14.113Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"5e29639260171c777c996aae","created_at":"2026-11-13T10:40:13.697Z","trace_propagation_headers":{"sentry-trace":"e26f107a95474185bf2c205248c944b5-6bd9f592984c4b2c","baggage":"sentry-trace_id=e26f107a95474185bf2c205248c944b5,sentry-release=Unknown"},"correlation_id":"01M7A7WNVD33SDQXCEHX35CC1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:13.700Z","job_size_bytes":19,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-5e29639260171c777c996aae: start","job_status":"start","queue_duration_s":0.413065,"scheduling_latency_s":0.413084}2068gitlab # [ 405.504672] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:14.138Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"5e29639260171c777c996aae","created_at":"2026-11-13T10:40:13.697Z","trace_propagation_headers":{"sentry-trace":"e26f107a95474185bf2c205248c944b5-6bd9f592984c4b2c","baggage":"sentry-trace_id=e26f107a95474185bf2c205248c944b5,sentry-release=Unknown"},"correlation_id":"01M7A7WNVD33SDQXCEHX35CC1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:13.700Z","job_size_bytes":19,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-5e29639260171c777c996aae: done: 0.025525 sec","job_status":"done","queue_duration_s":0.413065,"scheduling_latency_s":0.413084,"redis_calls":8,"redis_duration_s":0.0014880000000000002,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001157,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000331,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016264,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025525,"completed_at":"2026-11-13T10:40:14.138Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004481,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2069gitlab # [ 405.548904] gitaly[905]: time="2026-11-13T10:40:13.919Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WQGZ89A2PTSACNS1509B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:40:43.818" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:13.919" grpc.time_ms=0.641 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2070gitlab # [ 405.561865] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:14.140Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"a3dda7697e87e1c13f76fc39","created_at":"2026-11-13T10:40:13.700Z","trace_propagation_headers":{"sentry-trace":"e26f107a95474185bf2c205248c944b5-6bd9f592984c4b2c","baggage":"sentry-trace_id=e26f107a95474185bf2c205248c944b5,sentry-release=Unknown"},"correlation_id":"01M7A7WNVD33SDQXCEHX35CC1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:13.702Z","job_size_bytes":66,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-a3dda7697e87e1c13f76fc39: start","job_status":"start","queue_duration_s":0.437446,"scheduling_latency_s":0.437464}2071gitlab # [ 405.579530] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:14.163Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"a3dda7697e87e1c13f76fc39","created_at":"2026-11-13T10:40:13.700Z","trace_propagation_headers":{"sentry-trace":"e26f107a95474185bf2c205248c944b5-6bd9f592984c4b2c","baggage":"sentry-trace_id=e26f107a95474185bf2c205248c944b5,sentry-release=Unknown"},"correlation_id":"01M7A7WNVD33SDQXCEHX35CC1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:13.702Z","job_size_bytes":66,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-a3dda7697e87e1c13f76fc39: done: 0.023004 sec","job_status":"done","queue_duration_s":0.437446,"scheduling_latency_s":0.437464,"redis_calls":7,"redis_duration_s":0.001242,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000891,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000351,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015506,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023004,"completed_at":"2026-11-13T10:40:14.163Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003307,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2072gitlab # [ 405.624063] gitaly[905]: time="2026-11-13T10:40:14.435Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7WQ8XNXM66JRPDK5TD9N4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-13T10:40:44.134" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:14.434" grpc.time_ms=0.973 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2073gitlab # [ 405.636804] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:14.164Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"576eaa8d8453520f5511f1a8","created_at":"2026-11-13T10:40:13.703Z","trace_propagation_headers":{"sentry-trace":"e26f107a95474185bf2c205248c944b5-6bd9f592984c4b2c","baggage":"sentry-trace_id=e26f107a95474185bf2c205248c944b5,sentry-release=Unknown"},"correlation_id":"01M7A7WNVD33SDQXCEHX35CC1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:13.704Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-576eaa8d8453520f5511f1a8: start","job_status":"start","queue_duration_s":0.45985,"scheduling_latency_s":0.459866}2074gitlab # [ 405.653547] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:14.189Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"576eaa8d8453520f5511f1a8","created_at":"2026-11-13T10:40:13.703Z","trace_propagation_headers":{"sentry-trace":"e26f107a95474185bf2c205248c944b5-6bd9f592984c4b2c","baggage":"sentry-trace_id=e26f107a95474185bf2c205248c944b5,sentry-release=Unknown"},"correlation_id":"01M7A7WNVD33SDQXCEHX35CC1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:13.704Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-576eaa8d8453520f5511f1a8: done: 0.024564 sec","job_status":"done","queue_duration_s":0.45985,"scheduling_latency_s":0.459866,"redis_calls":7,"redis_duration_s":0.001198,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000865,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000333,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016126,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024564,"completed_at":"2026-11-13T10:40:14.189Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004548,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2075gitlab # [ 405.696909] gitaly[905]: time="2026-11-13T10:40:14.473Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WQ8XNXM66JRPDK5TD9N4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:40:44.172" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:14.472" grpc.time_ms=0.769 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2076gitlab # [ 405.709580] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:14.190Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"50126779021a8b54de4392f4","created_at":"2026-11-13T10:40:13.705Z","trace_propagation_headers":{"sentry-trace":"e26f107a95474185bf2c205248c944b5-6bd9f592984c4b2c","baggage":"sentry-trace_id=e26f107a95474185bf2c205248c944b5,sentry-release=Unknown"},"correlation_id":"01M7A7WNVD33SDQXCEHX35CC1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:13.706Z","job_size_bytes":6,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-50126779021a8b54de4392f4: start","job_status":"start","queue_duration_s":0.483823,"scheduling_latency_s":0.48384}2077gitlab # [ 405.726534] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:14.204Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"50126779021a8b54de4392f4","created_at":"2026-11-13T10:40:13.705Z","trace_propagation_headers":{"sentry-trace":"e26f107a95474185bf2c205248c944b5-6bd9f592984c4b2c","baggage":"sentry-trace_id=e26f107a95474185bf2c205248c944b5,sentry-release=Unknown"},"correlation_id":"01M7A7WNVD33SDQXCEHX35CC1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:13.706Z","job_size_bytes":6,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-50126779021a8b54de4392f4: done: 0.013921 sec","job_status":"done","queue_duration_s":0.483823,"scheduling_latency_s":0.48384,"redis_calls":7,"redis_duration_s":0.0012439999999999999,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000865,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000379,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009055,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013921,"completed_at":"2026-11-13T10:40:14.204Z","load_balancing_strategy":"primary","db_duration_s":0.000458,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2078gitlab # [ 405.769859] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:14.205Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"21b8234219cd80b996e6cd64","created_at":"2026-11-13T10:40:13.707Z","trace_propagation_headers":{"sentry-trace":"e26f107a95474185bf2c205248c944b5-6bd9f592984c4b2c","baggage":"sentry-trace_id=e26f107a95474185bf2c205248c944b5,sentry-release=Unknown"},"correlation_id":"01M7A7WNVD33SDQXCEHX35CC1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:13.708Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-21b8234219cd80b996e6cd64: start","job_status":"start","queue_duration_s":0.497156,"scheduling_latency_s":0.497173}2079gitlab # [ 405.785495] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:14.228Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"21b8234219cd80b996e6cd64","created_at":"2026-11-13T10:40:13.707Z","trace_propagation_headers":{"sentry-trace":"e26f107a95474185bf2c205248c944b5-6bd9f592984c4b2c","baggage":"sentry-trace_id=e26f107a95474185bf2c205248c944b5,sentry-release=Unknown"},"correlation_id":"01M7A7WNVD33SDQXCEHX35CC1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:13.708Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-21b8234219cd80b996e6cd64: done: 0.022627 sec","job_status":"done","queue_duration_s":0.497156,"scheduling_latency_s":0.497173,"redis_calls":7,"redis_duration_s":0.001238,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000888,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00035,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015483,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022627,"completed_at":"2026-11-13T10:40:14.228Z","load_balancing_strategy":"primary","db_duration_s":0.003336,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2080gitlab # [ 405.827707] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:14.229Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"4a29d464256d79e6a730091a","created_at":"2026-11-13T10:40:13.769Z","trace_propagation_headers":{"sentry-trace":"e26f107a95474185bf2c205248c944b5-6bd9f592984c4b2c","baggage":"sentry-trace_id=e26f107a95474185bf2c205248c944b5,sentry-release=Unknown"},"correlation_id":"01M7A7WNVD33SDQXCEHX35CC1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:13.771Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-4a29d464256d79e6a730091a: start","job_status":"start","queue_duration_s":0.457769,"scheduling_latency_s":0.457786}2081gitlab # [ 405.843741] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:14.258Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"4a29d464256d79e6a730091a","created_at":"2026-11-13T10:40:13.769Z","trace_propagation_headers":{"sentry-trace":"e26f107a95474185bf2c205248c944b5-6bd9f592984c4b2c","baggage":"sentry-trace_id=e26f107a95474185bf2c205248c944b5,sentry-release=Unknown"},"correlation_id":"01M7A7WNVD33SDQXCEHX35CC1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:13.771Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-4a29d464256d79e6a730091a: done: 0.028639 sec","job_status":"done","queue_duration_s":0.457769,"scheduling_latency_s":0.457786,"redis_calls":8,"redis_duration_s":0.004701,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004374,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000327,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017588,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028639,"completed_at":"2026-11-13T10:40:14.258Z","load_balancing_strategy":"primary","db_duration_s":0.004064,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2082gitlab # [ 405.887083] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:14.259Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"37608cd7bbab96e1a2c23213","created_at":"2026-11-13T10:40:13.830Z","trace_propagation_headers":{"sentry-trace":"0082ad5dea6b455f951f6f18bb8c234e-7dd62150cff84992","baggage":"sentry-trace_id=0082ad5dea6b455f951f6f18bb8c234e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7WQ8XNXM66JRPDK5TD9N4","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:13.835Z","job_size_bytes":65,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-37608cd7bbab96e1a2c23213: start","job_status":"start","queue_duration_s":0.424223,"scheduling_latency_s":0.42424}2083gitlab # [ 405.913895] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:14.300Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"37608cd7bbab96e1a2c23213","created_at":"2026-11-13T10:40:13.830Z","trace_propagation_headers":{"sentry-trace":"0082ad5dea6b455f951f6f18bb8c234e-7dd62150cff84992","baggage":"sentry-trace_id=0082ad5dea6b455f951f6f18bb8c234e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7WQ8XNXM66JRPDK5TD9N4","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:13.835Z","job_size_bytes":65,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-37608cd7bbab96e1a2c23213: done: 0.041002 sec","job_status":"done","queue_duration_s":0.424223,"scheduling_latency_s":0.42424,"redis_calls":7,"redis_duration_s":0.010846999999999999,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.010178,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000669,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021549,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.041002,"completed_at":"2026-11-13T10:40:14.300Z","load_balancing_strategy":"primary","db_duration_s":0.006165,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2084gitlab # [ 405.980046] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:14.302Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"93f09dcddbe93618ed89ec50","created_at":"2026-11-13T10:40:13.836Z","trace_propagation_headers":{"sentry-trace":"0082ad5dea6b455f951f6f18bb8c234e-7dd62150cff84992","baggage":"sentry-trace_id=0082ad5dea6b455f951f6f18bb8c234e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7WQ8XNXM66JRPDK5TD9N4","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:13.839Z","job_size_bytes":65,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-93f09dcddbe93618ed89ec50: start","job_status":"start","queue_duration_s":0.462819,"scheduling_latency_s":0.462837}2085gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5/statuses2086gitlab # [ 405.999855] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:14.359Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"93f09dcddbe93618ed89ec50","created_at":"2026-11-13T10:40:13.836Z","trace_propagation_headers":{"sentry-trace":"0082ad5dea6b455f951f6f18bb8c234e-7dd62150cff84992","baggage":"sentry-trace_id=0082ad5dea6b455f951f6f18bb8c234e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7WQ8XNXM66JRPDK5TD9N4","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:13.839Z","job_size_bytes":65,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-93f09dcddbe93618ed89ec50: done: 0.056502 sec","job_status":"done","queue_duration_s":0.462819,"scheduling_latency_s":0.462837,"redis_calls":10,"redis_duration_s":0.0024939999999999997,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001444,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.00105,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035035,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.056502,"completed_at":"2026-11-13T10:40:14.359Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017392,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2087gitlab # [ 406.044191] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:14.360Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"376092888ef7306f3b76f37a","created_at":"2026-11-13T10:40:13.924Z","trace_propagation_headers":{"sentry-trace":"0082ad5dea6b455f951f6f18bb8c234e-7dd62150cff84992","baggage":"sentry-trace_id=0082ad5dea6b455f951f6f18bb8c234e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7WQ8XNXM66JRPDK5TD9N4","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:13.928Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-376092888ef7306f3b76f37a: start","job_status":"start","queue_duration_s":0.431736,"scheduling_latency_s":0.431754}2088gitlab # [ 406.060428] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:14.488Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"376092888ef7306f3b76f37a","created_at":"2026-11-13T10:40:13.924Z","trace_propagation_headers":{"sentry-trace":"0082ad5dea6b455f951f6f18bb8c234e-7dd62150cff84992","baggage":"sentry-trace_id=0082ad5dea6b455f951f6f18bb8c234e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7WQ8XNXM66JRPDK5TD9N4","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:13.928Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-376092888ef7306f3b76f37a: done: 0.127536 sec","job_status":"done","queue_duration_s":0.431736,"scheduling_latency_s":0.431754,"gitaly_calls":2,"gitaly_duration_s":0.00674,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.006802,"redis_read_bytes":79,"redis_write_bytes":5120,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000883,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001106,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000263,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1131,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00178,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000207,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000543,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001137,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.093855,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.09264082,"duration_s":0.127536,"completed_at":"2026-11-13T10:40:14.488Z","load_balancing_strategy":"primary","db_duration_s":0.025847,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2089gitlab # [ 406.113551] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:14.489Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"61e8529671b5ea00c580b61f","created_at":"2026-11-13T10:40:14.419Z","trace_propagation_headers":{"sentry-trace":"0082ad5dea6b455f951f6f18bb8c234e-cfa6fc7d32114cbf","baggage":"sentry-trace_id=0082ad5dea6b455f951f6f18bb8c234e,sentry-release=Unknown"},"correlation_id":"01M7A7WQ8XNXM66JRPDK5TD9N4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:14.421Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-61e8529671b5ea00c580b61f: start","job_status":"start","queue_duration_s":0.068723,"scheduling_latency_s":0.068747}2090gitlab # [ 406.130104] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:14.534Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"61e8529671b5ea00c580b61f","created_at":"2026-11-13T10:40:14.419Z","trace_propagation_headers":{"sentry-trace":"0082ad5dea6b455f951f6f18bb8c234e-cfa6fc7d32114cbf","baggage":"sentry-trace_id=0082ad5dea6b455f951f6f18bb8c234e,sentry-release=Unknown"},"correlation_id":"01M7A7WQ8XNXM66JRPDK5TD9N4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:14.421Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-61e8529671b5ea00c580b61f: done: 0.044838 sec","job_status":"done","queue_duration_s":0.068723,"scheduling_latency_s":0.068747,"redis_calls":7,"redis_duration_s":0.0015040000000000001,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001082,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000422,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034799,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.044838,"completed_at":"2026-11-13T10:40:14.534Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005387,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2091gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5/statuses, in 0.17 seconds)2092gitlab # [ 406.175525] gitaly[905]: time="2026-11-13T10:40:15.087Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WRN5AJH0RJ9B25Z8KFMJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:40:45.085" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:15.086" grpc.time_ms=0.883 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2093gitlab # [ 406.188989] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:40:15 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1142094gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5/statuses2095gitlab # [ 407.291595] gitaly[905]: time="2026-11-13T10:40:16.240Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WSS4KC3HXW550A49BZN2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:40:46.238" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:16.239" grpc.time_ms=0.91 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2096gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5/statuses, in 0.14 seconds)2097gitlab # [ 407.323627] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:40:16 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1072098gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5/statuses2099gitlab # [ 408.613773] gitaly[905]: time="2026-11-13T10:40:17.562Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WTWRENDYF4PXNS4J4AY8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:40:47.460" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:17.561" grpc.time_ms=0.892 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2100gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5/statuses, in 0.32 seconds)2101gitlab # [ 408.643199] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:40:17 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 2862102gitlab # [ 409.367264] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:18.315Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"33d96ae3a17d484710624af6","created_at":"2026-11-13T10:40:18.265Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8666f72ab297a81cbe577b7c99932950","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:18.313Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-33d96ae3a17d484710624af6: start","job_status":"start","queue_duration_s":0.002222,"scheduling_latency_s":0.002262}2103gitlab # [ 409.503022] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:18.451Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"33d96ae3a17d484710624af6","created_at":"2026-11-13T10:40:18.265Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8666f72ab297a81cbe577b7c99932950","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:18.313Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-33d96ae3a17d484710624af6: done: 0.13622 sec","job_status":"done","queue_duration_s":0.002222,"scheduling_latency_s":0.002262,"redis_calls":12,"redis_duration_s":0.028873,"redis_read_bytes":10,"redis_write_bytes":1576,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001674,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.019833,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.007366,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037828,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0135739,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.13622,"completed_at":"2026-11-13T10:40:18.451Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.018581,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2104gitlab # [ 409.570807] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:18.457Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"f95ebd5da616dd99c075c355","created_at":"2026-11-13T10:40:18.333Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e6cc3bf9b00eaeb236b12075eb193551","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:18.342Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-f95ebd5da616dd99c075c355: start","job_status":"start","queue_duration_s":0.114796,"scheduling_latency_s":0.114828}2105gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5/statuses2106gitlab # [ 409.689880] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:18.637Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"f95ebd5da616dd99c075c355","created_at":"2026-11-13T10:40:18.333Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e6cc3bf9b00eaeb236b12075eb193551","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:18.342Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-f95ebd5da616dd99c075c355: done: 0.179693 sec","job_status":"done","queue_duration_s":0.114796,"scheduling_latency_s":0.114828,"redis_calls":13,"redis_duration_s":0.025731,"redis_read_bytes":11,"redis_write_bytes":1780,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.004717,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.014588,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.006426,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.016,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.060821,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00184648,"exclusive_lock_hold_duration_s":0.09066532,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.179693,"completed_at":"2026-11-13T10:40:18.637Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.055253,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2107gitlab # [ 409.757813] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:18.647Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"aed5a38b7ac3abb2ab375185","created_at":"2026-11-13T10:40:18.355Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0cbe05f2b98d615c319af50ab6ee98ec","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:18.371Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-aed5a38b7ac3abb2ab375185: start","job_status":"start","queue_duration_s":0.275459,"scheduling_latency_s":0.275493}2108gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5/statuses, in 0.14 seconds)2109gitlab # [ 409.777647] gitaly[905]: time="2026-11-13T10:40:18.698Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WW64BXZ7ZDYYVWVBX53T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:40:48.596" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:18.697" grpc.time_ms=0.772 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2110gitlab # [ 409.792735] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:18.717Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"aed5a38b7ac3abb2ab375185","created_at":"2026-11-13T10:40:18.355Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0cbe05f2b98d615c319af50ab6ee98ec","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:18.371Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-aed5a38b7ac3abb2ab375185: done: 0.070172 sec","job_status":"done","queue_duration_s":0.275459,"scheduling_latency_s":0.275493,"redis_calls":11,"redis_duration_s":0.007798,"redis_read_bytes":9,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000465,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001965,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.005368,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020273,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00070252,"exclusive_lock_hold_duration_s":0.00916974,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.070172,"completed_at":"2026-11-13T10:40:18.717Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007552,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2111gitlab # [ 409.852576] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:18.719Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"f1fd17914f3fbe540f8b184c","created_at":"2026-11-13T10:40:18.376Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"24334ec9c4ae8c6c2c67000948fec5be","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:18.384Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-f1fd17914f3fbe540f8b184c: start","job_status":"start","queue_duration_s":0.33442,"scheduling_latency_s":0.334448}2112gitlab # [ 409.874521] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:40:18 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1032113gitlab # [ 409.878167] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:18.770Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"f1fd17914f3fbe540f8b184c","created_at":"2026-11-13T10:40:18.376Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"24334ec9c4ae8c6c2c67000948fec5be","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:18.384Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-f1fd17914f3fbe540f8b184c: done: 0.051126 sec","job_status":"done","queue_duration_s":0.33442,"scheduling_latency_s":0.334448,"redis_calls":11,"redis_duration_s":0.006873,"redis_read_bytes":9,"redis_write_bytes":1386,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.003961,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002509,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000403,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020992,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.051126,"completed_at":"2026-11-13T10:40:18.769Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002978,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2114gitlab # [ 409.929802] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:18.772Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"fbd482d1ba41ae4c4e97c098","created_at":"2026-11-13T10:40:18.394Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2a3a87ba773f6bc9246b15c58256f9bf","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:18.413Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-fbd482d1ba41ae4c4e97c098: start","job_status":"start","queue_duration_s":0.359179,"scheduling_latency_s":0.359206}2115gitlab # [ 409.944527] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:18.862Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"fbd482d1ba41ae4c4e97c098","created_at":"2026-11-13T10:40:18.394Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2a3a87ba773f6bc9246b15c58256f9bf","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:18.413Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-fbd482d1ba41ae4c4e97c098: done: 0.0896 sec","job_status":"done","queue_duration_s":0.359179,"scheduling_latency_s":0.359206,"redis_calls":9,"redis_duration_s":0.010092,"redis_read_bytes":7,"redis_write_bytes":1026,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00199,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006419,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001683,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022819,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0896,"completed_at":"2026-11-13T10:40:18.862Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015223,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2116gitlab # [ 409.987595] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:18.867Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"cc28e924640cd9c179bd5ad0","created_at":"2026-11-13T10:40:18.435Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ed4799032ec6d98426366409ee120a4b","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:18.448Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-cc28e924640cd9c179bd5ad0: start","job_status":"start","queue_duration_s":0.417992,"scheduling_latency_s":0.418021}2117gitlab # [ 410.095055] nix-daemon[3607]: accepted connection from pid 4721, user nixbot2118gitlab # [ 410.106174] nix-daemon[3607]: accepted connection from pid 4683, user nixbot2119gitlab # [ 410.122727] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:19.071Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"cc28e924640cd9c179bd5ad0","created_at":"2026-11-13T10:40:18.435Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ed4799032ec6d98426366409ee120a4b","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:18.448Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-cc28e924640cd9c179bd5ad0: done: 0.2046 sec","job_status":"done","queue_duration_s":0.417992,"scheduling_latency_s":0.418021,"redis_calls":11,"redis_duration_s":0.012635,"redis_read_bytes":9,"redis_write_bytes":1174,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002377,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00706,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000896,"redis_rate_limiting_write_bytes":51,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002302,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01844,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00155814,"extra.cells_stale_requests_cleanup_cron_worker.removed_count":0,"duration_s":0.2046,"completed_at":"2026-11-13T10:40:19.071Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000609,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2120gitlab # [ 410.187522] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:19.078Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"db21c8d75265f79ab2e0ca99","created_at":"2026-11-13T10:40:18.458Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"baa382659919c78fbbaaaf1fde93577f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:18.466Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-db21c8d75265f79ab2e0ca99: start","job_status":"start","queue_duration_s":0.61255,"scheduling_latency_s":0.612577}2121gitlab # [ 410.243388] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:19.192Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"db21c8d75265f79ab2e0ca99","created_at":"2026-11-13T10:40:18.458Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"baa382659919c78fbbaaaf1fde93577f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:18.466Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-db21c8d75265f79ab2e0ca99: done: 0.113296 sec","job_status":"done","queue_duration_s":0.61255,"scheduling_latency_s":0.612577,"redis_calls":11,"redis_duration_s":0.013206,"redis_read_bytes":9,"redis_write_bytes":1399,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002033,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.007177,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003996,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02306,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00183184,"exclusive_lock_hold_duration_s":0.03653216,"duration_s":0.113296,"completed_at":"2026-11-13T10:40:19.191Z","load_balancing_strategy":"primary","db_duration_s":0.0336,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2122gitlab # [ 410.301760] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:19.198Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"2d8a2580a78cf541f12c9bbb","created_at":"2026-11-13T10:40:18.469Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ffe2132d7b25e166809223c73927bb81","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:18.483Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-2d8a2580a78cf541f12c9bbb: start","job_status":"start","queue_duration_s":0.714175,"scheduling_latency_s":0.714202}2123gitlab # [ 410.340214] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:19.288Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"2d8a2580a78cf541f12c9bbb","created_at":"2026-11-13T10:40:18.469Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ffe2132d7b25e166809223c73927bb81","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:18.483Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-2d8a2580a78cf541f12c9bbb: done: 0.090699 sec","job_status":"done","queue_duration_s":0.714175,"scheduling_latency_s":0.714202,"redis_calls":9,"redis_duration_s":0.010434,"redis_read_bytes":7,"redis_write_bytes":1183,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002563,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004922,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002949,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02829,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.090699,"completed_at":"2026-11-13T10:40:19.288Z","load_balancing_strategy":"primary","db_duration_s":0.024736,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2124gitlab # [ 410.391612] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:19.296Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"341bb6b622d00620125cf34e","created_at":"2026-11-13T10:40:18.497Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b7ad8bd74b891c2facb040ebc7da22a5","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:18.507Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-341bb6b622d00620125cf34e: start","job_status":"start","queue_duration_s":0.788523,"scheduling_latency_s":0.788626}2125gitlab # [ 410.422460] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:19.369Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"341bb6b622d00620125cf34e","created_at":"2026-11-13T10:40:18.497Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b7ad8bd74b891c2facb040ebc7da22a5","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:18.507Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-341bb6b622d00620125cf34e: done: 0.073894 sec","job_status":"done","queue_duration_s":0.788523,"scheduling_latency_s":0.788626,"redis_calls":9,"redis_duration_s":0.002462,"redis_read_bytes":7,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000461,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001402,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000599,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028015,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.073894,"completed_at":"2026-11-13T10:40:19.369Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005683,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2126gitlab # [ 410.467886] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:19.376Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"71e73f9ed7aefb5585e49512","created_at":"2026-11-13T10:40:18.561Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c6dd13a0f7d3f71deb0c68a29c2320bb","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:18.572Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-71e73f9ed7aefb5585e49512: start","job_status":"start","queue_duration_s":0.803082,"scheduling_latency_s":0.803109}2127gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5/statuses2128gitlab # [ 410.787011] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:19.735Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"71e73f9ed7aefb5585e49512","created_at":"2026-11-13T10:40:18.561Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c6dd13a0f7d3f71deb0c68a29c2320bb","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:18.572Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-71e73f9ed7aefb5585e49512: done: 0.359927 sec","job_status":"done","queue_duration_s":0.803082,"scheduling_latency_s":0.803109,"redis_calls":564,"redis_duration_s":0.072858,"redis_read_bytes":562,"redis_write_bytes":49063,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001452,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00163,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.069776,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.32174,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.359927,"completed_at":"2026-11-13T10:40:19.735Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000614,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2129gitlab # [ 410.855490] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:19.737Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"0f82af6a9383160d3e84d915","created_at":"2026-11-13T10:40:18.576Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3f735061f0d9501032860e6d9870f032","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:18.593Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-0f82af6a9383160d3e84d915: start","job_status":"start","queue_duration_s":1.144115,"scheduling_latency_s":1.144136}2130gitlab # [ 410.875664] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:19.750Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}2131gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5/statuses, in 0.14 seconds)2132gitlab # [ 410.880224] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:19.763Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"0f82af6a9383160d3e84d915","created_at":"2026-11-13T10:40:18.576Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3f735061f0d9501032860e6d9870f032","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:18.593Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-0f82af6a9383160d3e84d915: done: 0.025751 sec","job_status":"done","queue_duration_s":1.144115,"scheduling_latency_s":1.144136,"redis_calls":9,"redis_duration_s":0.0022960000000000003,"redis_read_bytes":7,"redis_write_bytes":1105,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000639,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001221,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000436,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014437,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025751,"completed_at":"2026-11-13T10:40:19.762Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001227,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2133gitlab # [ 410.925427] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:19.765Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivateExpiredDeploymentsCronWorker","jid":"5793611a4c6cc88015058712","created_at":"2026-11-13T10:40:18.610Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3962bc846031145ab462e00407a2d454","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:787f72254f5c8a3553ab694c3728fa911b46196df537a2b1a46751f692c5ffa6","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:18.624Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivateExpiredDeploymentsCronWorker JID-5793611a4c6cc88015058712: start","job_status":"start","queue_duration_s":1.140352,"scheduling_latency_s":1.140377}2134gitlab # [ 410.939268] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:19.824Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivateExpiredDeploymentsCronWorker","jid":"5793611a4c6cc88015058712","created_at":"2026-11-13T10:40:18.610Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3962bc846031145ab462e00407a2d454","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:787f72254f5c8a3553ab694c3728fa911b46196df537a2b1a46751f692c5ffa6","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:18.624Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivateExpiredDeploymentsCronWorker JID-5793611a4c6cc88015058712: done: 0.059527 sec","job_status":"done","queue_duration_s":1.140352,"scheduling_latency_s":1.140377,"redis_calls":10,"redis_duration_s":0.002115,"redis_read_bytes":9,"redis_write_bytes":1187,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000392,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001333,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00039,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035184,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00027086,"extra.pages_deactivate_expired_deployments_cron_worker.deactivate_expired_pages_deployments":{"deactivated_deployments":0,"duration":0.00289362},"duration_s":0.059527,"completed_at":"2026-11-13T10:40:19.824Z","load_balancing_strategy":"primary","db_duration_s":0.016182,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2135gitlab # [ 410.983853] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:40:19 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 962136gitlab # [ 410.987016] gitaly[905]: time="2026-11-13T10:40:19.842Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WX9XQTXAQ41VGQWVMEZK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:40:49.740" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:19.840" grpc.time_ms=0.858 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2137gitlab # [ 411.000202] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:19.826Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Packages::Cleanup::DeleteOrphanedDependenciesWorker","jid":"d9d4f94942e619f63c1fe112","created_at":"2026-11-13T10:40:18.638Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"db70204bc23acc6b285f85e788726298","meta.root_caller_id":"Cronjob","meta.feature_category":"package_registry","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ae1976bd21c961bb04f5dedb6e2b63c4b50acd9c52a47e7573ade27e3c65ce36","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:18.653Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Packages::Cleanup::DeleteOrphanedDependenciesWorker JID-d9d4f94942e619f63c1fe112: start","job_status":"start","queue_duration_s":1.17326,"scheduling_latency_s":1.173279}2138gitlab # [ 411.015393] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:19.860Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Packages::Cleanup::DeleteOrphanedDependenciesWorker","jid":"d9d4f94942e619f63c1fe112","created_at":"2026-11-13T10:40:18.638Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"db70204bc23acc6b285f85e788726298","meta.root_caller_id":"Cronjob","meta.feature_category":"package_registry","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ae1976bd21c961bb04f5dedb6e2b63c4b50acd9c52a47e7573ade27e3c65ce36","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:18.653Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Packages::Cleanup::DeleteOrphanedDependenciesWorker JID-d9d4f94942e619f63c1fe112: done: 0.033951 sec","job_status":"done","queue_duration_s":1.17326,"scheduling_latency_s":1.173279,"redis_calls":13,"redis_duration_s":0.0030359999999999996,"redis_read_bytes":10,"redis_write_bytes":1343,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000387,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":345,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001192,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.001457,"redis_shared_state_read_bytes":4,"redis_shared_state_write_bytes":368,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019884,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034118,"extra.packages_cleanup_delete_orphaned_dependencies_worker.last_processed_packages_dependency_id":0,"extra.packages_cleanup_delete_orphaned_dependencies_worker.deleted_rows_count":0,"duration_s":0.033951,"completed_at":"2026-11-13T10:40:19.860Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004945,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2139gitlab # [ 411.062898] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:19.862Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::InactiveProjectsDeletionCronWorker","jid":"91a663cdc7834ac4699d470f","created_at":"2026-11-13T10:40:18.681Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b2c5ba2cb282453daa95f9568709a3ed","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:442251d7c2d07bf110c698169f390fde7a2ca5a69ba3f639f0fb0d3b8866112f","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:18.691Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::InactiveProjectsDeletionCronWorker JID-91a663cdc7834ac4699d470f: start","job_status":"start","queue_duration_s":1.170463,"scheduling_latency_s":1.17049}2140gitlab # [ 411.077381] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:19.891Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::InactiveProjectsDeletionCronWorker","jid":"91a663cdc7834ac4699d470f","created_at":"2026-11-13T10:40:18.681Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b2c5ba2cb282453daa95f9568709a3ed","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:442251d7c2d07bf110c698169f390fde7a2ca5a69ba3f639f0fb0d3b8866112f","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:18.691Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::InactiveProjectsDeletionCronWorker JID-91a663cdc7834ac4699d470f: done: 0.029298 sec","job_status":"done","queue_duration_s":1.170463,"scheduling_latency_s":1.17049,"redis_calls":10,"redis_duration_s":0.0028280000000000002,"redis_read_bytes":9,"redis_write_bytes":1179,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000386,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00135,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":610,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001092,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":238,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019261,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00039286,"duration_s":0.029298,"completed_at":"2026-11-13T10:40:19.891Z","load_balancing_strategy":"primary","db_duration_s":0.001021,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2141gitlab # [ 411.119940] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:19.892Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"23105126fa744509fd457b93","created_at":"2026-11-13T10:40:18.696Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0a58b88d815e88f4c00eed4e26182141","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:18.708Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-23105126fa744509fd457b93: start","job_status":"start","queue_duration_s":1.184604,"scheduling_latency_s":1.184623}2142gitlab # [ 411.133786] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:19.913Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"23105126fa744509fd457b93","created_at":"2026-11-13T10:40:18.696Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0a58b88d815e88f4c00eed4e26182141","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:18.708Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-23105126fa744509fd457b93: done: 0.020438 sec","job_status":"done","queue_duration_s":1.184604,"scheduling_latency_s":1.184623,"redis_calls":11,"redis_duration_s":0.001854,"redis_read_bytes":8,"redis_write_bytes":1094,"redis_cache_calls":2,"redis_cache_duration_s":0.000341,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000337,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000853,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":482,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000323,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011949,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020438,"completed_at":"2026-11-13T10:40:19.913Z","load_balancing_strategy":"primary","db_duration_s":0.000365,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2143gitlab # [ 411.176728] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:19.914Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"674e095db773c678be11e160","created_at":"2026-11-13T10:40:18.715Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cd305279f6ac69d0bccc5052195fb9f5","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:18.726Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-674e095db773c678be11e160: start","job_status":"start","queue_duration_s":1.188049,"scheduling_latency_s":1.188066}2144gitlab # [ 411.190467] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:19.941Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"674e095db773c678be11e160","created_at":"2026-11-13T10:40:18.715Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cd305279f6ac69d0bccc5052195fb9f5","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:18.726Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-674e095db773c678be11e160: done: 0.027172 sec","job_status":"done","queue_duration_s":1.188049,"scheduling_latency_s":1.188066,"redis_calls":10,"redis_duration_s":0.0016849999999999999,"redis_read_bytes":9,"redis_write_bytes":1071,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00038,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000966,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000339,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016525,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00024344,"duration_s":0.027172,"completed_at":"2026-11-13T10:40:19.941Z","load_balancing_strategy":"primary","db_duration_s":0.004122,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2145gitlab # [ 411.232754] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:19.942Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"5e1b51a2acfa15e51c32fd68","created_at":"2026-11-13T10:40:18.748Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bdf103e641a50d1d0591d81f5fd87543","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-13T10:40:18.767Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-5e1b51a2acfa15e51c32fd68: start","job_status":"start","queue_duration_s":1.175213,"scheduling_latency_s":1.175228}2146gitlab # [ 411.247236] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:19.975Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"5e1b51a2acfa15e51c32fd68","created_at":"2026-11-13T10:40:18.748Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bdf103e641a50d1d0591d81f5fd87543","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-13T10:40:18.767Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-5e1b51a2acfa15e51c32fd68: done: 0.032796 sec","job_status":"done","queue_duration_s":1.175213,"scheduling_latency_s":1.175228,"redis_calls":9,"redis_duration_s":0.001891,"redis_read_bytes":7,"redis_write_bytes":1068,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000644,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000914,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000333,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021599,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032796,"completed_at":"2026-11-13T10:40:19.975Z","load_balancing_strategy":"primary","db_duration_s":0.003547,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2147gitlab # [ 411.289238] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:19.976Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainSslRenewalCronWorker","jid":"81eb2a7562267c11a04c89c7","created_at":"2026-11-13T10:40:18.784Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"59c6eb125f36357bfd6d5d79f710616d","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:cee20cd61cae5986aa71b6fa6dd981aaba3c2a4e4980220c0e15411712f7b4e3","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:18.792Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainSslRenewalCronWorker JID-81eb2a7562267c11a04c89c7: start","job_status":"start","queue_duration_s":1.184289,"scheduling_latency_s":1.184304}2148gitlab # [ 411.302781] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:19.998Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainSslRenewalCronWorker","jid":"81eb2a7562267c11a04c89c7","created_at":"2026-11-13T10:40:18.784Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"59c6eb125f36357bfd6d5d79f710616d","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:cee20cd61cae5986aa71b6fa6dd981aaba3c2a4e4980220c0e15411712f7b4e3","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:18.792Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainSslRenewalCronWorker JID-81eb2a7562267c11a04c89c7: done: 0.021246 sec","job_status":"done","queue_duration_s":1.184289,"scheduling_latency_s":1.184304,"redis_calls":10,"redis_duration_s":0.00164,"redis_read_bytes":9,"redis_write_bytes":1087,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000359,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000949,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":566,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000332,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":216,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013853,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025086,"duration_s":0.021246,"completed_at":"2026-11-13T10:40:19.998Z","load_balancing_strategy":"primary","db_duration_s":0.000551,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2149gitlab # [ 411.344771] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:19.999Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivatedDeploymentsDeleteCronWorker","jid":"c4f7b1983ec8f1adf701396e","created_at":"2026-11-13T10:40:18.808Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ff9cffc49f6b935aaaae06cac2f1f98b","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:02dc87c01bb5b1cecbe6eae0009120537d166f4adbbb83ef540bec12f0f9136a","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:18.820Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivatedDeploymentsDeleteCronWorker JID-c4f7b1983ec8f1adf701396e: start","job_status":"start","queue_duration_s":1.178945,"scheduling_latency_s":1.178961}2150gitlab # [ 411.358816] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:20.019Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivatedDeploymentsDeleteCronWorker","jid":"c4f7b1983ec8f1adf701396e","created_at":"2026-11-13T10:40:18.808Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ff9cffc49f6b935aaaae06cac2f1f98b","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:02dc87c01bb5b1cecbe6eae0009120537d166f4adbbb83ef540bec12f0f9136a","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:18.820Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivatedDeploymentsDeleteCronWorker JID-c4f7b1983ec8f1adf701396e: done: 0.019992 sec","job_status":"done","queue_duration_s":1.178945,"scheduling_latency_s":1.178961,"redis_calls":10,"redis_duration_s":0.001849,"redis_read_bytes":9,"redis_write_bytes":1187,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000385,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001019,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000445,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012136,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002566,"duration_s":0.019992,"completed_at":"2026-11-13T10:40:20.019Z","load_balancing_strategy":"primary","db_duration_s":0.001093,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2151gitlab # [ 411.401662] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:20.020Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"493ceec302945aa057a5c70e","created_at":"2026-11-13T10:40:18.831Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c76afd904ea5992b0a9b190b9aaea2e6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-13T10:40:18.840Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-493ceec302945aa057a5c70e: start","job_status":"start","queue_duration_s":1.180358,"scheduling_latency_s":1.180373}2152gitlab # [ 411.415474] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:20.031Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}2153gitlab # [ 411.419031] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:20.038Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"493ceec302945aa057a5c70e","created_at":"2026-11-13T10:40:18.831Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c76afd904ea5992b0a9b190b9aaea2e6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-13T10:40:18.840Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-493ceec302945aa057a5c70e: done: 0.018347 sec","job_status":"done","queue_duration_s":1.180358,"scheduling_latency_s":1.180373,"redis_calls":9,"redis_duration_s":0.001554,"redis_read_bytes":7,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000347,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000883,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000324,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01048,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018347,"completed_at":"2026-11-13T10:40:20.038Z","load_balancing_strategy":"primary","db_duration_s":0.000379,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2154gitlab # [ 411.462565] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:20.040Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::TimedOutBuilds::DropTimedOutWorker","jid":"94630a757ccb3d7d07fed0b1","created_at":"2026-11-13T10:40:18.850Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f57cdc7e3d426f6d555ff039d530653d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:11a5fdbfa1bf52035908acc769acf4caef1368e7f031a24ef8a306464e46866a","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:18.864Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropTimedOutWorker JID-94630a757ccb3d7d07fed0b1: start","job_status":"start","queue_duration_s":1.176455,"scheduling_latency_s":1.176472}2155gitlab # [ 411.477517] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:20.071Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::TimedOutBuilds::DropTimedOutWorker","jid":"94630a757ccb3d7d07fed0b1","created_at":"2026-11-13T10:40:18.850Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f57cdc7e3d426f6d555ff039d530653d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:11a5fdbfa1bf52035908acc769acf4caef1368e7f031a24ef8a306464e46866a","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:18.864Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropTimedOutWorker JID-94630a757ccb3d7d07fed0b1: done: 0.031261 sec","job_status":"done","queue_duration_s":1.176455,"scheduling_latency_s":1.176472,"redis_calls":20,"redis_duration_s":0.003275,"redis_read_bytes":23,"redis_write_bytes":4316,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001033,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1065,"redis_queues_calls":4,"redis_queues_duration_s":0.000491,"redis_queues_read_bytes":6,"redis_queues_write_bytes":2021,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.001389,"redis_queues_metadata_read_bytes":8,"redis_queues_metadata_write_bytes":1004,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000362,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":226,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02184,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0005128,"duration_s":0.031261,"completed_at":"2026-11-13T10:40:20.071Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001198,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2156gitlab # [ 411.522209] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:20.073Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"99e9d45cc81ed4d58da89452","created_at":"2026-11-13T10:40:18.870Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9a932f83b67e5c580943cc4bc29a605c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:18.877Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-99e9d45cc81ed4d58da89452: start","job_status":"start","queue_duration_s":1.195344,"scheduling_latency_s":1.19536}2157gitlab # [ 411.536113] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:20.106Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"99e9d45cc81ed4d58da89452","created_at":"2026-11-13T10:40:18.870Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9a932f83b67e5c580943cc4bc29a605c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:18.877Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-99e9d45cc81ed4d58da89452: done: 0.033193 sec","job_status":"done","queue_duration_s":1.195344,"scheduling_latency_s":1.19536,"redis_calls":18,"redis_duration_s":0.003154,"redis_read_bytes":20,"redis_write_bytes":4702,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000718,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000283,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.0016,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000553,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023692,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033193,"completed_at":"2026-11-13T10:40:20.106Z","load_balancing_strategy":"primary","db_duration_s":0.002067,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2158gitlab # [ 411.579557] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:20.107Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"47e4b88677db800a1973d829","created_at":"2026-11-13T10:40:18.882Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2b0a3d4eada7534de9811cc490ed8926","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:18.894Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-47e4b88677db800a1973d829: start","job_status":"start","queue_duration_s":1.213061,"scheduling_latency_s":1.213077}2159gitlab # [ 411.594792] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:20.147Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"47e4b88677db800a1973d829","created_at":"2026-11-13T10:40:18.882Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2b0a3d4eada7534de9811cc490ed8926","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:18.894Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-47e4b88677db800a1973d829: done: 0.040012 sec","job_status":"done","queue_duration_s":1.213061,"scheduling_latency_s":1.213077,"redis_calls":11,"redis_duration_s":0.0023020000000000002,"redis_read_bytes":9,"redis_write_bytes":1518,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000355,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000947,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02013,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0006585,"exclusive_lock_hold_duration_s":0.01446312,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.040012,"completed_at":"2026-11-13T10:40:20.147Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013848,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2160gitlab # [ 411.640717] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:20.148Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"57754b1fa44c11b9ff9193f3","created_at":"2026-11-13T10:40:18.907Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8be1ece19ca97f169d5d5c91ce9f8a2a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-13T10:40:19.068Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-57754b1fa44c11b9ff9193f3: start","job_status":"start","queue_duration_s":1.080389,"scheduling_latency_s":1.080406}2161gitlab # [ 411.655545] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:20.160Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}2162gitlab # [ 411.659594] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:20.211Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"57754b1fa44c11b9ff9193f3","created_at":"2026-11-13T10:40:18.907Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8be1ece19ca97f169d5d5c91ce9f8a2a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-13T10:40:19.068Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-57754b1fa44c11b9ff9193f3: done: 0.06254 sec","job_status":"done","queue_duration_s":1.080389,"scheduling_latency_s":1.080406,"redis_calls":9,"redis_duration_s":0.037104,"redis_read_bytes":7,"redis_write_bytes":1152,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00037,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003895,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.032839,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013356,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.06254,"completed_at":"2026-11-13T10:40:20.211Z","load_balancing_strategy":"primary","db_duration_s":0.000485,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2163gitlab # [ 411.701796] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:20.218Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"c29984868afb811753380c21","created_at":"2026-11-13T10:40:19.076Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8ea4760618689dbf39c083e3e4e8cb70","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:19.086Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-c29984868afb811753380c21: start","job_status":"start","queue_duration_s":1.132097,"scheduling_latency_s":1.132122}2164gitlab # [ 411.716716] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:20.269Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}2165gitlab # [ 411.720107] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:20.293Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"c29984868afb811753380c21","created_at":"2026-11-13T10:40:19.076Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8ea4760618689dbf39c083e3e4e8cb70","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:19.086Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-c29984868afb811753380c21: done: 0.075781 sec","job_status":"done","queue_duration_s":1.132097,"scheduling_latency_s":1.132122,"redis_calls":9,"redis_duration_s":0.011357,"redis_read_bytes":7,"redis_write_bytes":1185,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00176,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006797,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0028,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018883,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.075781,"completed_at":"2026-11-13T10:40:20.293Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002843,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2166gitlab # [ 411.763495] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:20.300Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"fff2024dc669df026065fe7e","created_at":"2026-11-13T10:40:19.097Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"30f94367a644f5006daf4cf0f10044c8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:19.111Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-fff2024dc669df026065fe7e: start","job_status":"start","queue_duration_s":1.188608,"scheduling_latency_s":1.188637}2167gitlab # [ 411.778615] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:20.376Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"fff2024dc669df026065fe7e","created_at":"2026-11-13T10:40:19.097Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"30f94367a644f5006daf4cf0f10044c8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:19.111Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-fff2024dc669df026065fe7e: done: 0.076322 sec","job_status":"done","queue_duration_s":1.188608,"scheduling_latency_s":1.188637,"redis_calls":9,"redis_duration_s":0.012462,"redis_read_bytes":7,"redis_write_bytes":1192,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003523,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.007288,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001651,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019267,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.076322,"completed_at":"2026-11-13T10:40:20.376Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002226,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2168gitlab # [ 411.822186] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:20.385Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"3b068bf2df2415f36c50a358","created_at":"2026-11-13T10:40:19.124Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"76dcfab50da903a2ee83425dfa64fa25","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:19.131Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-3b068bf2df2415f36c50a358: start","job_status":"start","queue_duration_s":1.254282,"scheduling_latency_s":1.254315}2169gitlab # [ 411.836600] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:20.473Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"3b068bf2df2415f36c50a358","created_at":"2026-11-13T10:40:19.124Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"76dcfab50da903a2ee83425dfa64fa25","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:19.131Z","job_size_bytes":3,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-3b068bf2df2415f36c50a358: done: 0.087357 sec","job_status":"done","queue_duration_s":1.254282,"scheduling_latency_s":1.254315,"redis_calls":11,"redis_duration_s":0.020416,"redis_read_bytes":137,"redis_write_bytes":1261,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.005034,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":395,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.008707,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.006675,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019795,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0020422,"duration_s":0.087357,"completed_at":"2026-11-13T10:40:20.473Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000835,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2170gitlab # [ 411.880223] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:20.480Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"6f17b6b139197130ddc2b4b2","created_at":"2026-11-13T10:40:19.134Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d6ac8fc247351613ddc40b7d63ab13e8","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:19.150Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-6f17b6b139197130ddc2b4b2: start","job_status":"start","queue_duration_s":1.330166,"scheduling_latency_s":1.33019}2171gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5/statuses2172gitlab # [ 411.894941] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:20.584Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"6f17b6b139197130ddc2b4b2","created_at":"2026-11-13T10:40:19.134Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d6ac8fc247351613ddc40b7d63ab13e8","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:19.150Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-6f17b6b139197130ddc2b4b2: done: 0.103744 sec","job_status":"done","queue_duration_s":1.330166,"scheduling_latency_s":1.33019,"redis_calls":11,"redis_duration_s":0.016891,"redis_read_bytes":9,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00208,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.008385,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.006426,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02681,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00267634,"exclusive_lock_hold_duration_s":0.0121373,"duration_s":0.103744,"completed_at":"2026-11-13T10:40:20.584Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011202,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2173gitlab # [ 411.937701] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:20.591Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"25d3577581f3799a4873c9ed","created_at":"2026-11-13T10:40:19.184Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d50090ba50fc2894eef2aa9520e234e7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:19.196Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-25d3577581f3799a4873c9ed: start","job_status":"start","queue_duration_s":1.394979,"scheduling_latency_s":1.395005}2174gitlab # [ 411.952323] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:20.650Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}2175gitlab # [ 411.955641] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:20.676Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"25d3577581f3799a4873c9ed","created_at":"2026-11-13T10:40:19.184Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d50090ba50fc2894eef2aa9520e234e7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:19.196Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-25d3577581f3799a4873c9ed: done: 0.084969 sec","job_status":"done","queue_duration_s":1.394979,"scheduling_latency_s":1.395005,"redis_calls":9,"redis_duration_s":0.014403000000000001,"redis_read_bytes":7,"redis_write_bytes":1112,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004035,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006739,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003629,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020365,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.084969,"completed_at":"2026-11-13T10:40:20.676Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000878,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2176gitlab # [ 411.999336] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:20.685Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"0588f8e32bf30fc1b1e9c19a","created_at":"2026-11-13T10:40:19.207Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4a6dc4cb18fd33e44672ea07a3e5d1c4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:19.221Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-0588f8e32bf30fc1b1e9c19a: start","job_status":"start","queue_duration_s":1.464171,"scheduling_latency_s":1.464203}2177gitlab # [ 412.042863] gitaly[905]: time="2026-11-13T10:40:20.991Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WYDSZ4W665CHNXDKMCS0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:40:50.889" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:20.990" grpc.time_ms=0.994 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2178gitlab # [ 412.082190] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:40:21 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1082179gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5/statuses, in 0.16 seconds)2180gitlab # [ 412.399823] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:21.348Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"0588f8e32bf30fc1b1e9c19a","created_at":"2026-11-13T10:40:19.207Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4a6dc4cb18fd33e44672ea07a3e5d1c4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:19.221Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-0588f8e32bf30fc1b1e9c19a: done: 0.662886 sec","job_status":"done","queue_duration_s":1.464171,"scheduling_latency_s":1.464203,"redis_calls":13,"redis_duration_s":0.020745,"redis_read_bytes":11,"redis_write_bytes":1567,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.005795,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.008151,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.006799,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.114,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.114,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.290715,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0027661,"exclusive_lock_hold_duration_s":0.56734944,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.662886,"completed_at":"2026-11-13T10:40:21.348Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.334729,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2181gitlab # [ 412.468697] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:21.355Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"5d2c19b31e766a28720414e6","created_at":"2026-11-13T10:40:19.232Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"15260e30e56a5e966425a88fe8353ec0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:19.250Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-5d2c19b31e766a28720414e6: start","job_status":"start","queue_duration_s":2.10549,"scheduling_latency_s":2.105522}2182gitlab # [ 412.487311] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:21.404Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}2183gitlab # [ 412.491930] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:21.428Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"5d2c19b31e766a28720414e6","created_at":"2026-11-13T10:40:19.232Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"15260e30e56a5e966425a88fe8353ec0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:19.250Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-5d2c19b31e766a28720414e6: done: 0.072665 sec","job_status":"done","queue_duration_s":2.10549,"scheduling_latency_s":2.105522,"redis_calls":9,"redis_duration_s":0.014105,"redis_read_bytes":7,"redis_write_bytes":1178,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003597,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006834,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003674,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019409,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.072665,"completed_at":"2026-11-13T10:40:21.428Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000795,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2184gitlab # [ 412.539808] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:21.435Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"cf36b77858e259ead022ff49","created_at":"2026-11-13T10:40:19.265Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:19.284Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-cf36b77858e259ead022ff49: start","job_status":"start","queue_duration_s":2.150837,"scheduling_latency_s":2.150865}2185gitlab # [ 412.860572] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:21.809Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"cf36b77858e259ead022ff49","created_at":"2026-11-13T10:40:19.265Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:19.284Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-cf36b77858e259ead022ff49: done: 0.37358 sec","job_status":"done","queue_duration_s":2.150837,"scheduling_latency_s":2.150865,"redis_calls":70,"redis_duration_s":0.104853,"redis_read_bytes":118,"redis_write_bytes":42580,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.013198,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.002145,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.083938,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.005572,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.158774,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.37358,"completed_at":"2026-11-13T10:40:21.809Z","load_balancing_strategy":"primary","db_duration_s":0.007917,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2186gitlab # [ 412.929930] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:21.816Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"6925aae8d2c4d2e18f6e0138","created_at":"2026-11-13T10:40:19.292Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8801d6d7e84068b23b52279c14c236ba","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:19.300Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-6925aae8d2c4d2e18f6e0138: start","job_status":"start","queue_duration_s":2.515523,"scheduling_latency_s":2.515555}2187gitlab # [ 412.955010] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:21.903Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"6925aae8d2c4d2e18f6e0138","created_at":"2026-11-13T10:40:19.292Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-05f24872d2e54340","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8801d6d7e84068b23b52279c14c236ba","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:19.300Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-6925aae8d2c4d2e18f6e0138: done: 0.087445 sec","job_status":"done","queue_duration_s":2.515523,"scheduling_latency_s":2.515555,"redis_calls":9,"redis_duration_s":0.013468999999999998,"redis_read_bytes":7,"redis_write_bytes":984,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003547,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.007342,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00258,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025432,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.087445,"completed_at":"2026-11-13T10:40:21.903Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005533,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2188gitlab # [ 413.009901] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:21.910Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropRunningWorker","jid":"0ec3e678e7fb51473aece0b7","created_at":"2026-11-13T10:40:20.051Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-596c36df74354ac9","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"f57cdc7e3d426f6d555ff039d530653d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c1f34cecaf7af64c199dc18ea8fa612209c900930a454f797bf95dbf199c209c","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:20.059Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropRunningWorker JID-0ec3e678e7fb51473aece0b7: start","job_status":"start","queue_duration_s":1.851333,"scheduling_latency_s":1.851362}2189gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5/statuses2190gitlab # [ 413.062212] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:22.010Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropRunningWorker","jid":"0ec3e678e7fb51473aece0b7","created_at":"2026-11-13T10:40:20.051Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-596c36df74354ac9","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"f57cdc7e3d426f6d555ff039d530653d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c1f34cecaf7af64c199dc18ea8fa612209c900930a454f797bf95dbf199c209c","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:20.059Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropRunningWorker JID-0ec3e678e7fb51473aece0b7: done: 0.1 sec","job_status":"done","queue_duration_s":1.851333,"scheduling_latency_s":1.851362,"redis_calls":10,"redis_duration_s":0.01124,"redis_read_bytes":9,"redis_write_bytes":1117,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001495,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.007445,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0023,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":222,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.022,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025195,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0015963,"duration_s":0.1,"completed_at":"2026-11-13T10:40:22.010Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.033306,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2191gitlab # [ 413.130193] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:22.018Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropCancelingWorker","jid":"18e4a011315cd0e3e2e3637d","created_at":"2026-11-13T10:40:20.060Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-596c36df74354ac9","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"f57cdc7e3d426f6d555ff039d530653d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:23125b5131e5285ca63ba8858fc6fb370306fe43f055d056b2a837c90773932d","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:20.064Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropCancelingWorker JID-18e4a011315cd0e3e2e3637d: start","job_status":"start","queue_duration_s":1.953673,"scheduling_latency_s":1.953701}2192gitlab # [ 413.164636] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:22.113Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropCancelingWorker","jid":"18e4a011315cd0e3e2e3637d","created_at":"2026-11-13T10:40:20.060Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-596c36df74354ac9","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"f57cdc7e3d426f6d555ff039d530653d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:23125b5131e5285ca63ba8858fc6fb370306fe43f055d056b2a837c90773932d","size_limiter":"validated","enqueued_at":"2026-11-13T10:40:20.064Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropCancelingWorker JID-18e4a011315cd0e3e2e3637d: done: 0.09513 sec","job_status":"done","queue_duration_s":1.953673,"scheduling_latency_s":1.953701,"redis_calls":10,"redis_duration_s":0.004573,"redis_read_bytes":9,"redis_write_bytes":1133,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002382,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001328,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":586,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000863,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":226,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023434,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00033142,"duration_s":0.09513,"completed_at":"2026-11-13T10:40:22.113Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006642,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2193gitlab # [ 413.222628] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:22.121Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"4014da91c5cfe6993ce8f71e","created_at":"2026-11-13T10:40:20.087Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-524bc22bb9ea40ad","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9a932f83b67e5c580943cc4bc29a605c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:20.099Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-4014da91c5cfe6993ce8f71e: start","job_status":"start","queue_duration_s":2.021833,"scheduling_latency_s":2.021865}2194gitlab # [ 413.237259] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:22.180Z","message":"No existing merge request to be cleaned up."}2195gitlab # [ 413.272752] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:22.221Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"4014da91c5cfe6993ce8f71e","created_at":"2026-11-13T10:40:20.087Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-524bc22bb9ea40ad","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9a932f83b67e5c580943cc4bc29a605c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:20.099Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-4014da91c5cfe6993ce8f71e: done: 0.100073 sec","job_status":"done","queue_duration_s":2.021833,"scheduling_latency_s":2.021865,"redis_calls":12,"redis_duration_s":0.019824,"redis_read_bytes":10,"redis_write_bytes":1323,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001996,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.007046,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.010782,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.007,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034578,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.100073,"completed_at":"2026-11-13T10:40:22.221Z","load_balancing_strategy":"primary","db_duration_s":0.021408,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2196gitlab # [ 413.332545] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:22.230Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"57462e42fd2652ca9419592e","created_at":"2026-11-13T10:40:20.087Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-524bc22bb9ea40ad","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9a932f83b67e5c580943cc4bc29a605c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:20.099Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-57462e42fd2652ca9419592e: start","job_status":"start","queue_duration_s":2.131061,"scheduling_latency_s":2.131098}2197gitlab # [ 413.346716] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:22.293Z","message":"No existing merge request to be cleaned up."}2198gitlab # [ 413.348951] gitaly[905]: time="2026-11-13T10:40:22.286Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7WZHZWDTNP51GCD8YW7HQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:40:52.284" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:22.285" grpc.time_ms=0.866 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2199gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5/statuses, in 0.29 seconds)2200gitlab # [ 413.371565] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:40:22 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 2412201gitlab # [ 413.374864] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:22.320Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"57462e42fd2652ca9419592e","created_at":"2026-11-13T10:40:20.087Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-524bc22bb9ea40ad","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9a932f83b67e5c580943cc4bc29a605c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:20.099Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-57462e42fd2652ca9419592e: done: 0.090196 sec","job_status":"done","queue_duration_s":2.131061,"scheduling_latency_s":2.131098,"redis_calls":10,"redis_duration_s":0.01011,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00389,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00622,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.009,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.009,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024155,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.090196,"completed_at":"2026-11-13T10:40:22.320Z","load_balancing_strategy":"primary","db_duration_s":0.022892,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2202gitlab # [ 413.417500] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:22.322Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"403e60dfc57994933a261421","created_at":"2026-11-13T10:40:20.087Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-524bc22bb9ea40ad","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9a932f83b67e5c580943cc4bc29a605c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:20.099Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-403e60dfc57994933a261421: start","job_status":"start","queue_duration_s":2.223256,"scheduling_latency_s":2.223304}2203gitlab # [ 413.430181] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:22.361Z","message":"No existing merge request to be cleaned up."}2204gitlab # [ 413.441711] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:22.390Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"403e60dfc57994933a261421","created_at":"2026-11-13T10:40:20.087Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-524bc22bb9ea40ad","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9a932f83b67e5c580943cc4bc29a605c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:20.099Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-403e60dfc57994933a261421: done: 0.06791 sec","job_status":"done","queue_duration_s":2.223256,"scheduling_latency_s":2.223304,"redis_calls":10,"redis_duration_s":0.013999,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004451,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.009548,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022483,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.06791,"completed_at":"2026-11-13T10:40:22.390Z","load_balancing_strategy":"primary","db_duration_s":0.011506,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2205gitlab # [ 413.482598] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:22.398Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"522e472a36d0dd8a33a9d9dc","created_at":"2026-11-13T10:40:20.087Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-524bc22bb9ea40ad","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9a932f83b67e5c580943cc4bc29a605c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:20.099Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-522e472a36d0dd8a33a9d9dc: start","job_status":"start","queue_duration_s":2.298847,"scheduling_latency_s":2.298872}2206gitlab # [ 413.501546] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:22.450Z","message":"No existing merge request to be cleaned up."}2207gitlab # [ 413.531936] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:22.480Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"522e472a36d0dd8a33a9d9dc","created_at":"2026-11-13T10:40:20.087Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-524bc22bb9ea40ad","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9a932f83b67e5c580943cc4bc29a605c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:20.099Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-522e472a36d0dd8a33a9d9dc: done: 0.082604 sec","job_status":"done","queue_duration_s":2.298847,"scheduling_latency_s":2.298872,"redis_calls":10,"redis_duration_s":0.014124,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.005363,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.008761,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022499,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.082604,"completed_at":"2026-11-13T10:40:22.480Z","load_balancing_strategy":"primary","db_duration_s":0.007923,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2208gitlab # [ 413.582320] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:22.487Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eda7880b0cc21325d530b9c4","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eda7880b0cc21325d530b9c4: start","job_status":"start","queue_duration_s":0.706336,"scheduling_latency_s":0.706363}2209gitlab # [ 413.626190] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:22.574Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eda7880b0cc21325d530b9c4","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eda7880b0cc21325d530b9c4: done: 0.08706 sec","job_status":"done","queue_duration_s":0.706336,"scheduling_latency_s":0.706363,"redis_calls":15,"redis_duration_s":0.022869,"redis_read_bytes":12,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003676,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.005489,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.013704,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023874,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.08706,"completed_at":"2026-11-13T10:40:22.574Z","load_balancing_strategy":"primary","db_duration_s":0.000782,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2210gitlab # [ 413.690321] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:22.582Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d9a624932a510dcafa65271","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d9a624932a510dcafa65271: start","job_status":"start","queue_duration_s":0.800703,"scheduling_latency_s":0.800739}2211gitlab # [ 413.713969] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:22.662Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d9a624932a510dcafa65271","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d9a624932a510dcafa65271: done: 0.080414 sec","job_status":"done","queue_duration_s":0.800703,"scheduling_latency_s":0.800739,"redis_calls":13,"redis_duration_s":0.019119999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.006791,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.012329,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018266,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.080414,"completed_at":"2026-11-13T10:40:22.662Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2212gitlab # [ 413.776091] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:22.669Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7fadb64fdb8f682c10150325","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7fadb64fdb8f682c10150325: start","job_status":"start","queue_duration_s":0.887963,"scheduling_latency_s":0.887988}2213gitlab # [ 413.791638] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:22.738Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7fadb64fdb8f682c10150325","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7fadb64fdb8f682c10150325: done: 0.069489 sec","job_status":"done","queue_duration_s":0.887963,"scheduling_latency_s":0.887988,"redis_calls":13,"redis_duration_s":0.018771,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.007341,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.01143,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017933,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.069489,"completed_at":"2026-11-13T10:40:22.738Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2214gitlab # [ 413.833065] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:22.745Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8f558fa47c6b8ca00ecd2f49","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8f558fa47c6b8ca00ecd2f49: start","job_status":"start","queue_duration_s":0.963619,"scheduling_latency_s":0.963645}2215gitlab # [ 413.870274] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:22.818Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8f558fa47c6b8ca00ecd2f49","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8f558fa47c6b8ca00ecd2f49: done: 0.074005 sec","job_status":"done","queue_duration_s":0.963619,"scheduling_latency_s":0.963645,"redis_calls":13,"redis_duration_s":0.017780999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004978,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.012803,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017855,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.074005,"completed_at":"2026-11-13T10:40:22.818Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2216gitlab # [ 413.923091] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:22.826Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4605ae3098a4da5c0733c4d5","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4605ae3098a4da5c0733c4d5: start","job_status":"start","queue_duration_s":1.044611,"scheduling_latency_s":1.044636}2217gitlab # [ 413.947954] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:22.896Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4605ae3098a4da5c0733c4d5","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4605ae3098a4da5c0733c4d5: done: 0.070576 sec","job_status":"done","queue_duration_s":1.044611,"scheduling_latency_s":1.044636,"redis_calls":13,"redis_duration_s":0.018145,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.005019,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.013126,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017606,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.070576,"completed_at":"2026-11-13T10:40:22.896Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2218gitlab # [ 413.993984] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:22.903Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"651825d058459a4c790f35f1","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-651825d058459a4c790f35f1: start","job_status":"start","queue_duration_s":1.122332,"scheduling_latency_s":1.122359}2219gitlab # [ 414.176302] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.124Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"651825d058459a4c790f35f1","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-651825d058459a4c790f35f1: done: 0.221295 sec","job_status":"done","queue_duration_s":1.122332,"scheduling_latency_s":1.122359,"redis_calls":13,"redis_duration_s":0.01624,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.005832,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.010408,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016429,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.221295,"completed_at":"2026-11-13T10:40:23.124Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2220gitlab # [ 414.238181] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.131Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2700daac0bc03587f626f7f6","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2700daac0bc03587f626f7f6: start","job_status":"start","queue_duration_s":1.35038,"scheduling_latency_s":1.350414}2221gitlab # [ 414.263037] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.211Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2700daac0bc03587f626f7f6","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2700daac0bc03587f626f7f6: done: 0.080026 sec","job_status":"done","queue_duration_s":1.35038,"scheduling_latency_s":1.350414,"redis_calls":13,"redis_duration_s":0.019726,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.005962,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.013764,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018606,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.080026,"completed_at":"2026-11-13T10:40:23.211Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2222gitlab # [ 414.314446] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.217Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"39351e206d0c8d530d67013f","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-39351e206d0c8d530d67013f: start","job_status":"start","queue_duration_s":1.436467,"scheduling_latency_s":1.436492}2223gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5/statuses2224gitlab # [ 414.354855] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.302Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"39351e206d0c8d530d67013f","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-39351e206d0c8d530d67013f: done: 0.084672 sec","job_status":"done","queue_duration_s":1.436467,"scheduling_latency_s":1.436492,"redis_calls":13,"redis_duration_s":0.021036,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.006398,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.014638,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018157,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.084672,"completed_at":"2026-11-13T10:40:23.302Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2225gitlab # [ 414.405151] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.307Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"28a00d0600b1fff1c94c58bf","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-28a00d0600b1fff1c94c58bf: start","job_status":"start","queue_duration_s":1.526193,"scheduling_latency_s":1.526222}2226gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5/statuses, in 0.15 seconds)2227gitlab # [ 414.484230] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.432Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"28a00d0600b1fff1c94c58bf","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-28a00d0600b1fff1c94c58bf: done: 0.124691 sec","job_status":"done","queue_duration_s":1.526193,"scheduling_latency_s":1.526222,"redis_calls":13,"redis_duration_s":0.0288,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.01376,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.01504,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018483,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.124691,"completed_at":"2026-11-13T10:40:23.432Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2228gitlab # [ 414.541973] gitaly[905]: time="2026-11-13T10:40:23.435Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7X0T4X68SAJ1Y5K3H2KQG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:40:53.433" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:23.433" grpc.time_ms=1.102 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2229gitlab # [ 414.558134] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:40:23 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1062230gitlab # [ 414.566125] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.515Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"21d401b3f6a812f2c941b5da","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-21d401b3f6a812f2c941b5da: start","job_status":"start","queue_duration_s":1.733613,"scheduling_latency_s":1.733644}2231gitlab # [ 414.632851] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.581Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"21d401b3f6a812f2c941b5da","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-21d401b3f6a812f2c941b5da: done: 0.06663 sec","job_status":"done","queue_duration_s":1.733613,"scheduling_latency_s":1.733644,"redis_calls":13,"redis_duration_s":0.014971,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003863,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.011108,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02052,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.06663,"completed_at":"2026-11-13T10:40:23.581Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2232gitlab # [ 414.691549] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.587Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ff4f2c643982d203cdcc24bb","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ff4f2c643982d203cdcc24bb: start","job_status":"start","queue_duration_s":1.806529,"scheduling_latency_s":1.806559}2233gitlab # [ 414.706313] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.616Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ff4f2c643982d203cdcc24bb","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ff4f2c643982d203cdcc24bb: done: 0.02842 sec","job_status":"done","queue_duration_s":1.806529,"scheduling_latency_s":1.806559,"redis_calls":13,"redis_duration_s":0.003665,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000964,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002701,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016047,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02842,"completed_at":"2026-11-13T10:40:23.616Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2234gitlab # [ 414.748172] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.618Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4512f2c88a7bc388aa489a10","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4512f2c88a7bc388aa489a10: start","job_status":"start","queue_duration_s":1.836679,"scheduling_latency_s":1.836704}2235gitlab # [ 414.760666] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.640Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4512f2c88a7bc388aa489a10","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4512f2c88a7bc388aa489a10: done: 0.02208 sec","job_status":"done","queue_duration_s":1.836679,"scheduling_latency_s":1.836704,"redis_calls":13,"redis_duration_s":0.0023480000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000791,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001557,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015268,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02208,"completed_at":"2026-11-13T10:40:23.640Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2236gitlab # [ 414.801339] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.641Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"878be6853dc9e17416fc944b","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-878be6853dc9e17416fc944b: start","job_status":"start","queue_duration_s":1.860195,"scheduling_latency_s":1.860217}2237gitlab # [ 414.814086] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.659Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"878be6853dc9e17416fc944b","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-878be6853dc9e17416fc944b: done: 0.017746 sec","job_status":"done","queue_duration_s":1.860195,"scheduling_latency_s":1.860217,"redis_calls":13,"redis_duration_s":0.001907,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000691,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001216,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012266,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017746,"completed_at":"2026-11-13T10:40:23.659Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2238gitlab # [ 414.854138] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.661Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f12482151ed54829d6082557","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f12482151ed54829d6082557: start","job_status":"start","queue_duration_s":1.88043,"scheduling_latency_s":1.880451}2239gitlab # [ 414.866755] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.678Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f12482151ed54829d6082557","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f12482151ed54829d6082557: done: 0.016642 sec","job_status":"done","queue_duration_s":1.88043,"scheduling_latency_s":1.880451,"redis_calls":13,"redis_duration_s":0.0030419999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000668,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002374,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011382,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016642,"completed_at":"2026-11-13T10:40:23.678Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2240gitlab # [ 414.907137] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.679Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"29375a7217fabfb064a56fa5","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-29375a7217fabfb064a56fa5: start","job_status":"start","queue_duration_s":1.898382,"scheduling_latency_s":1.8984}2241gitlab # [ 414.919839] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.695Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"29375a7217fabfb064a56fa5","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-29375a7217fabfb064a56fa5: done: 0.016253 sec","job_status":"done","queue_duration_s":1.898382,"scheduling_latency_s":1.8984,"redis_calls":13,"redis_duration_s":0.001816,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000679,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001137,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010637,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016253,"completed_at":"2026-11-13T10:40:23.695Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2242gitlab # [ 414.960863] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.697Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"448497948c90bc7777835ae5","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-448497948c90bc7777835ae5: start","job_status":"start","queue_duration_s":1.915752,"scheduling_latency_s":1.915768}2243gitlab # [ 414.973590] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.713Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"448497948c90bc7777835ae5","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-448497948c90bc7777835ae5: done: 0.01673 sec","job_status":"done","queue_duration_s":1.915752,"scheduling_latency_s":1.915768,"redis_calls":13,"redis_duration_s":0.002055,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000729,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001326,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011665,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01673,"completed_at":"2026-11-13T10:40:23.713Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2244gitlab # [ 415.013252] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.716Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e39502500653bd780d34f50b","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e39502500653bd780d34f50b: start","job_status":"start","queue_duration_s":1.935179,"scheduling_latency_s":1.935198}2245gitlab # [ 415.025769] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.732Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e39502500653bd780d34f50b","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e39502500653bd780d34f50b: done: 0.01587 sec","job_status":"done","queue_duration_s":1.935179,"scheduling_latency_s":1.935198,"redis_calls":13,"redis_duration_s":0.001872,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00064,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001232,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010829,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01587,"completed_at":"2026-11-13T10:40:23.732Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2246gitlab # [ 415.065810] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.734Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2233ce5b0724a81c0345a4ed","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2233ce5b0724a81c0345a4ed: start","job_status":"start","queue_duration_s":1.953027,"scheduling_latency_s":1.953053}2247gitlab # [ 415.078502] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.757Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2233ce5b0724a81c0345a4ed","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2233ce5b0724a81c0345a4ed: done: 0.022662 sec","job_status":"done","queue_duration_s":1.953027,"scheduling_latency_s":1.953053,"redis_calls":13,"redis_duration_s":0.002454,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000833,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001621,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016035,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022662,"completed_at":"2026-11-13T10:40:23.756Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2248gitlab # [ 415.118668] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.760Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"319ed5885e498f1a1c99009b","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-319ed5885e498f1a1c99009b: start","job_status":"start","queue_duration_s":1.97879,"scheduling_latency_s":1.978812}2249gitlab # [ 415.131914] nix-daemon[3607]: accepted connection from pid 4683, user nixbot2250gitlab # [ 415.132974] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.782Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"319ed5885e498f1a1c99009b","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-319ed5885e498f1a1c99009b: done: 0.022152 sec","job_status":"done","queue_duration_s":1.97879,"scheduling_latency_s":1.978812,"redis_calls":13,"redis_duration_s":0.002399,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000787,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001612,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015187,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022152,"completed_at":"2026-11-13T10:40:23.782Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2251gitlab # [ 415.173825] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.783Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"29b987c94fc5b4aa788742cd","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-29b987c94fc5b4aa788742cd: start","job_status":"start","queue_duration_s":2.002461,"scheduling_latency_s":2.002481}2252gitlab # [ 415.186730] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.807Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"29b987c94fc5b4aa788742cd","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-29b987c94fc5b4aa788742cd: done: 0.023793 sec","job_status":"done","queue_duration_s":2.002461,"scheduling_latency_s":2.002481,"redis_calls":13,"redis_duration_s":0.002777,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001113,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001664,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016448,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023793,"completed_at":"2026-11-13T10:40:23.807Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2253gitlab # [ 415.226686] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.809Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2fa67f0bb2342e42e45ff3f6","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2fa67f0bb2342e42e45ff3f6: start","job_status":"start","queue_duration_s":2.027768,"scheduling_latency_s":2.027791}2254gitlab # [ 415.239450] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.830Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2fa67f0bb2342e42e45ff3f6","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2fa67f0bb2342e42e45ff3f6: done: 0.021417 sec","job_status":"done","queue_duration_s":2.027768,"scheduling_latency_s":2.027791,"redis_calls":13,"redis_duration_s":0.0022389999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000789,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00145,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014983,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021417,"completed_at":"2026-11-13T10:40:23.830Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2255gitlab # [ 415.279557] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.833Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b82504f5c558e8e6742bece","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b82504f5c558e8e6742bece: start","job_status":"start","queue_duration_s":2.051779,"scheduling_latency_s":2.051801}2256gitlab # [ 415.292412] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.853Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b82504f5c558e8e6742bece","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b82504f5c558e8e6742bece: done: 0.020129 sec","job_status":"done","queue_duration_s":2.051779,"scheduling_latency_s":2.051801,"redis_calls":13,"redis_duration_s":0.0022570000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000745,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001512,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014029,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020129,"completed_at":"2026-11-13T10:40:23.853Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2257gitlab # [ 415.332581] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.854Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bcba7a2ce35c1fe8072c4460","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bcba7a2ce35c1fe8072c4460: start","job_status":"start","queue_duration_s":2.073314,"scheduling_latency_s":2.073332}2258gitlab # [ 415.345976] gitaly[905]: time="2026-11-13T10:40:24.206Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7X1JC9FDP15GY07FBMB9W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:40:54.204" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:24.205" grpc.time_ms=0.627 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2259gitlab # [ 415.359795] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.872Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bcba7a2ce35c1fe8072c4460","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bcba7a2ce35c1fe8072c4460: done: 0.017908 sec","job_status":"done","queue_duration_s":2.073314,"scheduling_latency_s":2.073332,"redis_calls":13,"redis_duration_s":0.0019160000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000671,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001245,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011662,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017908,"completed_at":"2026-11-13T10:40:23.872Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2260gitlab # [ 415.400972] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.873Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0cdc9fa2031768113de9d51c","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0cdc9fa2031768113de9d51c: start","job_status":"start","queue_duration_s":2.092595,"scheduling_latency_s":2.092616}2261gitlab # [ 415.413802] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:40:24 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 1612262gitlab # [ 415.416428] gitaly[905]: time="2026-11-13T10:40:24.250Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7X1JC9FDP15GY07FBMB9W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-13T10:40:34.001" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:40:24.242" grpc.time_ms=8.229 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2263gitlab # [ 415.430280] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.890Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0cdc9fa2031768113de9d51c","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0cdc9fa2031768113de9d51c: done: 0.01682 sec","job_status":"done","queue_duration_s":2.092595,"scheduling_latency_s":2.092616,"redis_calls":13,"redis_duration_s":0.002044,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000655,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001389,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010815,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01682,"completed_at":"2026-11-13T10:40:23.890Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2264gitlab # [ 415.470735] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.892Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ac158deeda2a14c7e0599ec1","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ac158deeda2a14c7e0599ec1: start","job_status":"start","queue_duration_s":2.111221,"scheduling_latency_s":2.111238}2265gitlab # [ 415.484134] gitaly[905]: time="2026-11-13T10:40:24.362Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7X1QM0RDMPT1M644H5EVA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:40:54.361" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:24.361" grpc.time_ms=0.774 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2266gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5/statuses2267gitlab # [ 415.497652] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.908Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ac158deeda2a14c7e0599ec1","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ac158deeda2a14c7e0599ec1: done: 0.015613 sec","job_status":"done","queue_duration_s":2.111221,"scheduling_latency_s":2.111238,"redis_calls":13,"redis_duration_s":0.001961,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000677,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001284,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010656,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015613,"completed_at":"2026-11-13T10:40:23.908Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2268gitlab # [ 415.537713] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.909Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dfac6aeede1651abaadeeaa2","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dfac6aeede1651abaadeeaa2: start","job_status":"start","queue_duration_s":2.127945,"scheduling_latency_s":2.127961}2269gitlab # [ 415.551391] gitaly[905]: time="2026-11-13T10:40:24.408Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01M7A7X1QM0RDMPT1M644H5EVA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-13T10:40:34.159" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:40:24.399" grpc.time_ms=9.104 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2270gitlab # [ 415.565957] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:40:24 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1522271gitlab # [ 415.568559] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.924Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dfac6aeede1651abaadeeaa2","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dfac6aeede1651abaadeeaa2: done: 0.01476 sec","job_status":"done","queue_duration_s":2.127945,"scheduling_latency_s":2.127961,"redis_calls":13,"redis_duration_s":0.001727,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000624,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001103,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010016,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01476,"completed_at":"2026-11-13T10:40:23.924Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2272gitlab # [ 415.608897] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.926Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5cf31ab41f67419fd983c931","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5cf31ab41f67419fd983c931: start","job_status":"start","queue_duration_s":2.144715,"scheduling_latency_s":2.144731}2273gitlab # [ 415.621580] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.941Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5cf31ab41f67419fd983c931","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5cf31ab41f67419fd983c931: done: 0.014996 sec","job_status":"done","queue_duration_s":2.144715,"scheduling_latency_s":2.144731,"redis_calls":13,"redis_duration_s":0.0017720000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000644,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001128,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010402,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014996,"completed_at":"2026-11-13T10:40:23.941Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2274gitlab # [ 415.661802] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.942Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"160eb0ad24b9dc9654c4ecd1","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-160eb0ad24b9dc9654c4ecd1: start","job_status":"start","queue_duration_s":2.160845,"scheduling_latency_s":2.16086}2275gitlab # [ 415.674667] gitaly[905]: time="2026-11-13T10:40:24.525Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7X1WQJZN4H1NE3WZAVDHB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:40:54.523" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:40:24.524" grpc.time_ms=0.821 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2276gitlab # [ 415.687965] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:40:24 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1452277gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5/statuses, in 0.19 seconds)2278(finished: subtest: eval and build post commit statuses, in 10.87 seconds)2279(finished: run the VM test script, in 485.66 seconds)2280gitlab # [ 415.690898] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.957Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"160eb0ad24b9dc9654c4ecd1","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-160eb0ad24b9dc9654c4ecd1: done: 0.014952 sec","job_status":"done","queue_duration_s":2.160845,"scheduling_latency_s":2.16086,"redis_calls":13,"redis_duration_s":0.001925,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000641,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001284,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010138,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014952,"completed_at":"2026-11-13T10:40:23.957Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2281gitlab # [ 415.733066] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.958Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8aa9c84bbf7cbf0080085311","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8aa9c84bbf7cbf0080085311: start","job_status":"start","queue_duration_s":2.17693,"scheduling_latency_s":2.176946}2282gitlab # [ 415.745859] gitaly[905]: time="2026-11-13T10:40:24.567Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M7A7X1WQJZN4H1NE3WZAVDHB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-13T10:40:34.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:40:24.559" grpc.time_ms=7.514 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2283gitlab # [ 415.763084] gitlab-workhorse[818]: gitlab 2001:db8:1::1 - - [2026/11/13:10:40:24 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5/statuses HTTP/1.1" 200 1334 "" "curl/8.21.0" 1132284test script finished in 485.73s2285cleanup2286kill QemuMachine (pid 46)2287gitlab # [ 415.766465] sidekiq-cluster[3321]: {"severity":"INFO","time":"2026-11-13T10:40:23.972Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8aa9c84bbf7cbf0080085311","created_at":"2026-11-13T10:40:21.505Z","trace_propagation_headers":{"sentry-trace":"cd2c838de4094c2b80b6fc21dd32f540-ab3584d612ca4c50","baggage":"sentry-trace_id=cd2c838de4094c2b80b6fc21dd32f540,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ec3f8723b9daf4f653dab1e7a716e128","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:40:21.781Z","job_size_bytes":2,"pid":3321,"sidekiq_tid":"m8i9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8aa9c84bbf7cbf0080085311: done: 0.014691 sec","job_status":"done","queue_duration_s":2.17693,"scheduling_latency_s":2.176946,"redis_calls":13,"redis_duration_s":0.001829,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000627,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001202,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_maqemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/41m77i1296n33p6liin8ynr6wh3h6b7m-python3-3.14.6/bin/python3.14)2288(finished: cleanup, in 1.58 seconds)2289additionally exposed symbols:2290 gitlab,2291 vlan1,2292 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh2293[]2294[]2295[{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-11-13T10:38:46.712Z', 'push_events': True, 'tag_push_events': True, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]2296[]2297[]2298[]2299[]2300[]2301[]2302[]2303[{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'pending', 'eval_warnings': None, 'error': None, 'created_at': '2026-11-13T10:40:13.587676Z', 'started_at': None, 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': {'position': 1, 'blocked_by': []}}]2304[{'id': 5, 'sha': '6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-13T10:40:13.918Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2305[{'id': 5, 'sha': '6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-13T10:40:13.918Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2306[{'id': 5, 'sha': '6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-13T10:40:13.918Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2307[{'id': 5, 'sha': '6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-13T10:40:13.918Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2308[{'id': 5, 'sha': '6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-13T10:40:13.918Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2309[{'id': 5, 'sha': '6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-13T10:40:13.918Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2310[{'id': 5, 'sha': '6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-13T10:40:13.918Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2311[{'id': 5, 'sha': '6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-13T10:40:13.918Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2312[{'id': 5, 'sha': '6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-13T10:40:13.918Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2313[{'id': 5, 'sha': '6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 9s (1 warning)', 'created_at': '2026-11-13T10:40:13.918Z', 'started_at': None, 'finished_at': '2026-11-13T10:40:24.273Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': '6f5cdc2f13a629d2087f97eaa9e9ef4cba3c82a5', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '0 attributes built', 'created_at': '2026-11-13T10:40:24.437Z', 'started_at': None, 'finished_at': '2026-11-13T10:40:24.590Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]