vm-test-run-nixbot-gitlab
checks.aarch64-linux.nixbot-gitlab
· build #394
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 3600 seconds6run the VM test script7start all VMs8gitlab: starting vm9gitlab: QEMU running (pid 46)10gitlab # Disk image does not exist, creating the virtualisation disk image...11gitlab # Formatting '/build/vm-state-gitlab/tmp.7SVUFxLiMW', fmt=raw size=107374182412gitlab # mke2fs 1.47.4 (6-Mar-2025)13gitlab # Discarding device blocks: 0/262144 done14gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes15gitlab # Filesystem UUID: b03b4e32-2437-44f6-91a3-d20db517cabe16(finished: start all VMs, in 66.73 seconds)17gitlab # Superblock backups stored on blocks:18gitlab: waiting for unit setup-gitlab.service19gitlab # 32768, 98304, 163840, 22937620gitlab: waiting for the VM to finish booting21gitlab # 22gitlab # Allocating group tables: 0/8 done23gitlab # Writing inode tables: 0/8 done24gitlab # Creating journal (8192 blocks): done25gitlab # Writing superblocks and filesystem accounting information: 0/8 done26gitlab # 27gitlab # Virtualisation disk image created.28gitlab # Creating Nix store image...29gitlab # Created Nix store image.30gitlab # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]31gitlab # [ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Thu Aug 6 17:27:09 UTC 202632gitlab # [ 0.000000] KASLR enabled33gitlab # [ 0.000000] random: crng init done34gitlab # [ 0.000000] Machine model: linux,dummy-virt35gitlab # [ 0.000000] efi: UEFI not found.36gitlab # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT37gitlab # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x00000001bfffffff]38gitlab # [ 0.000000] NODE_DATA(0) allocated [mem 0x1bfdeb740-0x1bfdeeebf]39gitlab # [ 0.000000] Zone ranges:40gitlab # [ 0.000000] DMA [mem 0x0000000040000000-0x00000000ffffffff]41gitlab # [ 0.000000] DMA32 empty42gitlab # [ 0.000000] Normal [mem 0x0000000100000000-0x00000001bfffffff]43gitlab # [ 0.000000] Device empty44gitlab # [ 0.000000] Movable zone start for each node45gitlab # [ 0.000000] Early memory node ranges46gitlab # [ 0.000000] node 0: [mem 0x0000000040000000-0x00000001bfffffff]47gitlab # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000001bfffffff]48gitlab # [ 0.000000] cma: Reserved 32 MiB at 0x00000000fe00000049gitlab # [ 0.000000] psci: probing for conduit method from DT.50gitlab # [ 0.000000] psci: PSCIv1.3 detected in firmware.51gitlab # [ 0.000000] psci: Using standard PSCI v0.2 function IDs52gitlab # [ 0.000000] psci: Trusted OS migration not required53gitlab # [ 0.000000] psci: SMC Calling Convention v1.154gitlab # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)55gitlab # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u31129656gitlab # [ 0.000000] Detected PIPT I-cache on CPU057gitlab # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)58gitlab # [ 0.000000] CPU features: detected: GICv3 CPU interface59gitlab # [ 0.000000] CPU features: detected: Spectre-v460gitlab # [ 0.000000] CPU features: detected: Spectre-BHB61gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_3862gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_2363gitlab # [ 0.000000] alternatives: applying boot alternatives64gitlab # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/6b0rfvdx2flkyw3l6cvkv2fm5h59ya45-nixos-system-gitlab-test/init regInfo=/nix/store/lhabjkcdg9ms8s7rhd5wym17846k7s52-closure-info/registration console=ttyAMA0,115200n8 console=tty065gitlab # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/lhabjkcdg9ms8s7rhd5wym17846k7s52-closure-info/registration", will be passed to user space.66gitlab # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes67gitlab # [ 0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)68gitlab # [ 0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)69gitlab # [ 0.000000] software IO TLB: area num 4.70gitlab # [ 0.000000] software IO TLB: mapped [mem 0x00000000fa000000-0x00000000fe000000] (64MB)71gitlab # [ 0.000000] Fallback order for Node 0: 072gitlab # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 157286473gitlab # [ 0.000000] Policy zone: Normal74gitlab # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off75gitlab # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=176gitlab # [ 0.000000] allocated 12582912 bytes of page_ext77gitlab # [ 0.000000] ftrace: allocating 74867 entries in 294 pages78gitlab # [ 0.000000] ftrace: allocated 294 pages with 4 groups79gitlab # [ 0.000000] rcu: Hierarchical RCU implementation.80gitlab # [ 0.000000] rcu: RCU event tracing is enabled.81gitlab # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.82gitlab # [ 0.000000] Trampoline variant of Tasks RCU enabled.83gitlab # [ 0.000000] Rude variant of Tasks RCU enabled.84gitlab # [ 0.000000] Tracing variant of Tasks RCU enabled.85gitlab # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.86gitlab # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=487gitlab # [ 0.000000] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.88gitlab # [ 0.000000] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.89gitlab # [ 0.000000] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.90gitlab # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 091gitlab # [ 0.000000] GICv3: 256 SPIs implemented92gitlab # [ 0.000000] GICv3: 0 Extended SPIs implemented93gitlab # [ 0.000000] Root IRQ handler: gic_handle_irq94gitlab # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI95gitlab # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=096gitlab # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a000097gitlab # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]98gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @100f10000 (indirect, esz 8, psz 64K, shr 1)99gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @100f20000 (flat, esz 8, psz 64K, shr 1)100gitlab # [ 0.000000] GICv3: using LPI property table @0x0000000100f40000101gitlab # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000100f50000102gitlab # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.103gitlab # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns104gitlab # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).105gitlab # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns106gitlab # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns107gitlab # [ 0.000029] arm-pv: using stolen time PV108gitlab # [ 0.000370] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)109gitlab # [ 0.000552] Console: colour dummy device 80x25110gitlab # [ 0.000560] printk: legacy console [tty0] enabled111gitlab # [ 0.000752] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)112gitlab # [ 0.000758] pid_max: default: 32768 minimum: 301113gitlab # [ 0.000838] LSM: initializing lsm=capability,landlock,yama,bpf,ima114gitlab # [ 0.000975] landlock: Up and running.115gitlab # [ 0.000978] Yama: becoming mindful.116gitlab # [ 0.001388] LSM support for eBPF active117gitlab # [ 0.001601] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)118gitlab # [ 0.001724] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)119gitlab # [ 0.002963] cacheinfo: Unable to detect cache hierarchy for CPU 0120gitlab # [ 0.003684] rcu: Hierarchical SRCU implementation.121gitlab # [ 0.003688] rcu: Max phase no-delay instances is 1000.122gitlab # [ 0.003844] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level123gitlab # [ 0.004974] fsl-mc MSI: its@8080000 domain created124gitlab # [ 0.005066] EFI services will not be available.125gitlab # [ 0.005264] smp: Bringing up secondary CPUs ...126gitlab # [ 0.005780] Detected PIPT I-cache on CPU1127gitlab # [ 0.005842] GICv3: CPU1: found redistributor 1 region 0:0x00000000080c0000128gitlab # [ 0.005914] GICv3: CPU1: using allocated LPI pending table @0x0000000100f60000129gitlab # [ 0.005988] CPU1: Booted secondary processor 0x0000000001 [0xc00fac40]130gitlab # [ 0.006866] Detected PIPT I-cache on CPU2131gitlab # [ 0.006930] GICv3: CPU2: found redistributor 2 region 0:0x00000000080e0000132gitlab # [ 0.007002] GICv3: CPU2: using allocated LPI pending table @0x0000000100f70000133gitlab # [ 0.007074] CPU2: Booted secondary processor 0x0000000002 [0xc00fac40]134gitlab # [ 0.007833] Detected PIPT I-cache on CPU3135gitlab # [ 0.007902] GICv3: CPU3: found redistributor 3 region 0:0x0000000008100000136gitlab # [ 0.007981] GICv3: CPU3: using allocated LPI pending table @0x0000000100f80000137gitlab # [ 0.008058] CPU3: Booted secondary processor 0x0000000003 [0xc00fac40]138gitlab # [ 0.008359] smp: Brought up 1 node, 4 CPUs139gitlab # [ 0.008385] SMP: Total of 4 processors activated.140gitlab # [ 0.008388] CPU: All CPU(s) started at EL1141gitlab # [ 0.008396] CPU features: detected: Branch Target Identification142gitlab # [ 0.008400] CPU features: detected: ARMv8.4 Translation Table Level143gitlab # [ 0.008402] CPU features: detected: Instruction cache invalidation not required for I/D coherence144gitlab # [ 0.008406] CPU features: detected: Data cache clean to the PoU not required for I/D coherence145gitlab # [ 0.008409] CPU features: detected: Common not Private translations146gitlab # [ 0.008412] CPU features: detected: CRC32 instructions147gitlab # [ 0.008414] CPU features: detected: Data cache clean to Point of Deep Persistence148gitlab # [ 0.008417] CPU features: detected: Data cache clean to Point of Persistence149gitlab # [ 0.008420] CPU features: detected: Data independent timing control (DIT)150gitlab # [ 0.008423] CPU features: detected: E0PD151gitlab # [ 0.008425] CPU features: detected: Enhanced Counter Virtualization152gitlab # [ 0.008428] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)153gitlab # [ 0.008430] CPU features: detected: Enhanced Virtualization Traps154gitlab # [ 0.008433] CPU features: detected: Fine Grained Traps155gitlab # [ 0.008436] CPU features: detected: Generic authentication (architected QARMA5 algorithm)156gitlab # [ 0.008439] CPU features: detected: RCpc load-acquire (LDAPR)157gitlab # [ 0.008442] CPU features: detected: LSE atomic instructions158gitlab # [ 0.008444] CPU features: detected: Privileged Access Never159gitlab # [ 0.008447] CPU features: detected: PMUv3160gitlab # [ 0.008449] CPU features: detected: RAS Extension Support161gitlab # [ 0.008451] CPU features: detected: RASv1p1 Extension Support162gitlab # [ 0.008454] CPU features: detected: Random Number Generator163gitlab # [ 0.008456] CPU features: detected: Speculation barrier (SB)164gitlab # [ 0.008458] CPU features: detected: Stage-2 Force Write-Back165gitlab # [ 0.008461] CPU features: detected: TLB range maintenance instructions166gitlab # [ 0.008464] CPU features: detected: Speculative Store Bypassing Safe (SSBS)167gitlab # [ 0.008550] alternatives: applying system-wide alternatives168gitlab # [ 0.011357] CPU features: detected: BBM Level 2 without TLB conflict abort169gitlab # [ 0.011565] Memory: 5963764K/6291456K available (24448K kernel code, 7090K rwdata, 26340K rodata, 4736K init, 1106K bss, 272908K reserved, 32768K cma-reserved)170gitlab # [ 0.012635] devtmpfs: initialized171gitlab # [ 0.016043] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)172gitlab # [ 0.016127] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).173gitlab # [ 0.016326] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL174gitlab # [ 0.016331] 0 pages in range for non-PLT usage175gitlab # [ 0.016332] 508288 pages in range for PLT usage176gitlab # [ 0.016447] pinctrl core: initialized pinctrl subsystem177gitlab # [ 0.017212] DMI not present or invalid.178gitlab # [ 0.020427] NET: Registered PF_NETLINK/PF_ROUTE protocol family179gitlab # [ 0.023211] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations180gitlab # [ 0.023463] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations181gitlab # [ 0.025389] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations182gitlab # [ 0.025411] audit: initializing netlink subsys (disabled)183gitlab # [ 0.025583] audit: type=2000 audit(0.024:1): state=initialized audit_enabled=0 res=1184gitlab # [ 0.026024] thermal_sys: Registered thermal governor 'fair_share'185gitlab # [ 0.026026] thermal_sys: Registered thermal governor 'bang_bang'186gitlab # [ 0.026030] thermal_sys: Registered thermal governor 'step_wise'187gitlab # [ 0.026033] thermal_sys: Registered thermal governor 'user_space'188gitlab # [ 0.026035] thermal_sys: Registered thermal governor 'power_allocator'189gitlab # [ 0.026090] cpuidle: using governor ladder190gitlab # [ 0.026109] cpuidle: using governor menu191gitlab # [ 0.026314] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.192gitlab # [ 0.026437] ASID allocator initialised with 65536 entries193gitlab # [ 0.027626] Serial: AMBA PL011 UART driver194gitlab # [ 0.032813] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1195gitlab # [ 0.033033] printk: console [ttyAMA0] enabled196gitlab # [ 0.039051] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages197gitlab # [ 0.039064] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page198gitlab # [ 0.039068] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages199gitlab # [ 0.039073] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page200gitlab # [ 0.039077] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages201gitlab # [ 0.039081] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page202gitlab # [ 0.039086] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages203gitlab # [ 0.039090] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page204gitlab # [ 0.046782] fbcon: Taking over console205gitlab # [ 0.046798] ACPI: Interpreter disabled.206gitlab # [ 0.048267] iommu: Default domain type: Translated207gitlab # [ 0.048276] iommu: DMA domain TLB invalidation policy: strict mode208gitlab # [ 0.060358] SCSI subsystem initialized209gitlab # [ 0.061385] usbcore: registered new interface driver usbfs210gitlab # [ 0.061433] usbcore: registered new interface driver hub211gitlab # [ 0.061457] usbcore: registered new device driver usb212gitlab # [ 0.061907] pps_core: LinuxPPS API ver. 1 registered213gitlab # [ 0.061912] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>214gitlab # [ 0.061924] PTP clock support registered215gitlab # [ 0.061980] EDAC MC: Ver: 3.0.0216gitlab # [ 0.062373] scmi_core: SCMI protocol bus registered217gitlab # [ 0.062966] FPGA manager framework218gitlab # [ 0.064094] vgaarb: loaded219gitlab # [ 0.064571] clocksource: Switched to clocksource arch_sys_counter220gitlab # [ 0.066794] VFS: Disk quotas dquot_6.6.0221gitlab # [ 0.066828] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)222gitlab # [ 0.067345] netfs: FS-Cache loaded223gitlab # [ 0.067470] pnp: PnP ACPI: disabled224gitlab # [ 0.074254] NET: Registered PF_INET protocol family225gitlab # [ 0.075956] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)226gitlab # [ 0.117866] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)227gitlab # [ 0.117930] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)228gitlab # [ 0.117989] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)229gitlab # [ 0.118391] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)230gitlab # [ 0.119175] TCP: Hash tables configured (established 65536 bind 65536)231gitlab # [ 0.119303] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)232gitlab # [ 0.119365] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)233gitlab # [ 0.119518] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)234gitlab # [ 0.119763] NET: Registered PF_UNIX/PF_LOCAL protocol family235gitlab # [ 0.119789] NET: Registered PF_XDP protocol family236gitlab # [ 0.119804] PCI: CLS 0 bytes, default 64237gitlab # [ 0.119962] Trying to unpack rootfs image as initramfs...238gitlab # [ 0.129701] kvm [1]: HYP mode not available239gitlab # [ 0.186264] Initialise system trusted keyrings240gitlab # [ 0.186478] workingset: timestamp_bits=42 max_order=21 bucket_order=0241gitlab # [ 0.187044] squashfs: version 4.0 (2009/01/31) Phillip Lougher242gitlab # [ 0.187145] 9p: Installing v9fs 9p2000 file system support243gitlab # [ 0.200805] Key type asymmetric registered244gitlab # [ 0.200811] Asymmetric key parser 'x509' registered245gitlab # [ 0.200860] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)246gitlab # [ 0.201005] io scheduler mq-deadline registered247gitlab # [ 0.201010] io scheduler kyber registered248gitlab # [ 0.215650] pl061_gpio 9030000.pl061: PL061 GPIO chip registered249gitlab # [ 0.224614] ledtrig-cpu: registered to indicate activity on CPUs250gitlab # [ 0.225107] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:251gitlab # [ 0.225122] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000252gitlab # [ 0.225131] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000253gitlab # [ 0.225137] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000254gitlab # [ 0.225157] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits255gitlab # [ 0.225177] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]256gitlab # [ 0.225264] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00257gitlab # [ 0.225271] pci_bus 0000:00: root bus resource [bus 00-ff]258gitlab # [ 0.225285] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]259gitlab # [ 0.225289] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]260gitlab # [ 0.225292] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]261gitlab # [ 0.225407] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint262gitlab # [ 0.225981] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint263gitlab # [ 0.226213] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]264gitlab # [ 0.226227] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]265gitlab # [ 0.226261] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]266gitlab # [ 0.226274] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]267gitlab # [ 0.226840] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint268gitlab # [ 0.227082] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]269gitlab # [ 0.227094] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]270gitlab # [ 0.227124] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]271gitlab # [ 0.227709] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint272gitlab # [ 0.227943] pci 0000:00:03.0: BAR 0 [io 0x0000-0x001f]273gitlab # [ 0.227956] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]274gitlab # [ 0.227985] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]275gitlab # [ 0.228558] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint276gitlab # [ 0.228815] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]277gitlab # [ 0.228828] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]278gitlab # [ 0.228857] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]279gitlab # [ 0.229429] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint280gitlab # [ 0.229669] pci 0000:00:05.0: BAR 0 [io 0x0000-0x007f]281gitlab # [ 0.229681] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]282gitlab # [ 0.229710] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]283gitlab # [ 0.230311] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint284gitlab # [ 0.230542] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]285gitlab # [ 0.230607] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]286gitlab # [ 0.230642] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]287gitlab # [ 0.231219] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint288gitlab # [ 0.231449] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]289gitlab # [ 0.231462] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]290gitlab # [ 0.231497] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]291gitlab # [ 0.231510] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]292gitlab # [ 0.232139] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint293gitlab # [ 0.232388] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]294gitlab # [ 0.232417] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]295gitlab # [ 0.233027] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint296gitlab # [ 0.233274] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]297gitlab # [ 0.233304] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]298gitlab # [ 0.233797] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint299gitlab # [ 0.234028] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]300gitlab # [ 0.234355] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint301gitlab # [ 0.234637] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]302gitlab # [ 0.234650] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]303gitlab # [ 0.234687] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]304gitlab # [ 0.235277] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint305gitlab # [ 0.235513] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]306gitlab # [ 0.235526] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]307gitlab # [ 0.235554] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]308gitlab # [ 0.236259] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned309gitlab # [ 0.236266] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned310gitlab # [ 0.236270] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned311gitlab # [ 0.236319] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned312gitlab # [ 0.236369] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned313gitlab # [ 0.236423] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned314gitlab # [ 0.236473] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned315gitlab # [ 0.236524] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned316gitlab # [ 0.236585] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned317gitlab # [ 0.236643] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned318gitlab # [ 0.236697] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned319gitlab # [ 0.236752] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned320gitlab # [ 0.236824] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned321gitlab # [ 0.236878] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned322gitlab # [ 0.236904] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned323gitlab # [ 0.236929] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned324gitlab # [ 0.236954] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned325gitlab # [ 0.236979] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned326gitlab # [ 0.237000] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned327gitlab # [ 0.237020] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned328gitlab # [ 0.237040] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned329gitlab # [ 0.237061] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned330gitlab # [ 0.237081] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned331gitlab # [ 0.237107] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned332gitlab # [ 0.237128] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned333gitlab # [ 0.237148] pci 0000:00:05.0: BAR 0 [io 0x1000-0x107f]: assigned334gitlab # [ 0.237173] pci 0000:00:06.0: BAR 0 [io 0x1080-0x10ff]: assigned335gitlab # [ 0.237197] pci 0000:00:0b.0: BAR 0 [io 0x1100-0x113f]: assigned336gitlab # [ 0.237222] pci 0000:00:01.0: BAR 0 [io 0x1140-0x115f]: assigned337gitlab # [ 0.237247] pci 0000:00:02.0: BAR 0 [io 0x1160-0x117f]: assigned338gitlab # [ 0.237267] pci 0000:00:03.0: BAR 0 [io 0x1180-0x119f]: assigned339gitlab # [ 0.237286] pci 0000:00:04.0: BAR 0 [io 0x11a0-0x11bf]: assigned340gitlab # [ 0.237306] pci 0000:00:07.0: BAR 0 [io 0x11c0-0x11df]: assigned341gitlab # [ 0.237330] pci 0000:00:0c.0: BAR 0 [io 0x11e0-0x11ff]: assigned342gitlab # [ 0.237359] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]343gitlab # [ 0.237365] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]344gitlab # [ 0.237368] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]345gitlab # [ 0.238930] pci 0000:00:0a.0: enabling device (0000 -> 0002)346gitlab # [ 0.252065] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)347gitlab # [ 0.254397] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)348gitlab # [ 0.256789] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)349gitlab # [ 0.259493] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)350gitlab # [ 0.261876] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)351gitlab # [ 0.266360] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)352gitlab # [ 0.268662] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)353gitlab # [ 0.271017] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)354gitlab # [ 0.273494] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)355gitlab # [ 0.275507] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)356gitlab # [ 0.280474] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)357gitlab # [ 0.286848] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled358gitlab # [ 0.289452] msm_serial: driver initialized359gitlab # [ 0.289689] SuperH (H)SCI(F) driver initialized360gitlab # [ 0.289763] STM32 USART driver initialized361gitlab # [ 0.322134] loop: module loaded362gitlab # [ 0.322443] virtio_blk virtio4: 4/0/0 default/read/poll queues363gitlab # [ 0.325590] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)364gitlab # [ 0.328259] virtio_blk virtio5: 4/0/0 default/read/poll queues365gitlab # [ 0.330963] virtio_blk virtio5: [vdb] 12285584 512-byte logical blocks (6.29 GB/5.86 GiB)366gitlab # [ 0.333772] megasas: 07.734.00.00-rc1367gitlab # [ 0.334712] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]368gitlab # [ 0.337361] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000369gitlab # [ 0.337455] Intel/Sharp Extended Query Table at 0x0031370gitlab # [ 0.339692] Using buffer write method371gitlab # [ 0.339734] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]372gitlab # [ 0.342215] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000373gitlab # [ 0.342233] Intel/Sharp Extended Query Table at 0x0031374gitlab # [ 0.344213] Using buffer write method375gitlab # [ 0.344231] Concatenating MTD devices:376gitlab # [ 0.344234] (0): "0.flash"377gitlab # [ 0.344237] (1): "0.flash"378gitlab # [ 0.344239] into device "0.flash"379gitlab # [ 0.587515] Freeing initrd memory: 27184K380gitlab # [ 0.597739] tun: Universal TUN/TAP device driver, 1.6381gitlab # [ 0.602771] thunder_xcv, ver 1.0382gitlab # [ 0.602835] thunder_bgx, ver 1.0383gitlab # [ 0.602896] nicpf, ver 1.0384gitlab # [ 0.603811] e1000: Intel(R) PRO/1000 Network Driver385gitlab # [ 0.603818] e1000: Copyright (c) 1999-2006 Intel Corporation.386gitlab # [ 0.603869] e1000e: Intel(R) PRO/1000 Network Driver387gitlab # [ 0.603877] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.388gitlab # [ 0.603913] igb: Intel(R) Gigabit Ethernet Network Driver389gitlab # [ 0.603918] igb: Copyright (c) 2007-2014 Intel Corporation.390gitlab # [ 0.603962] igbvf: Intel(R) Gigabit Virtual Function Network Driver391gitlab # [ 0.603967] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.392gitlab # [ 0.604184] sky2: driver version 1.30393gitlab # [ 0.606968] usbcore: registered new interface driver usb-storage394gitlab # [ 0.607323] usbcore: registered new interface driver usbserial_generic395gitlab # [ 0.607348] usbserial: USB Serial support registered for generic396gitlab # [ 0.607541] ehci-pci 0000:00:0a.0: EHCI Host Controller397gitlab # [ 0.607579] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 1398gitlab # [ 0.607999] ehci-pci 0000:00:0a.0: irq 15, io mem 0x10089000399gitlab # [ 0.608307] hv_vmbus: registering driver hyperv_keyboard400gitlab # [ 0.609732] rtc-pl031 9010000.pl031: registered as rtc0401gitlab # [ 0.609765] rtc-pl031 9010000.pl031: setting system clock to 2026-09-25T15:54:36 UTC (1790351676)402gitlab # [ 0.610226] i2c_dev: i2c /dev entries driver403gitlab # [ 0.614505] sdhci: Secure Digital Host Controller Interface driver404gitlab # [ 0.614513] sdhci: Copyright(c) Pierre Ossman405gitlab # [ 0.614987] Synopsys Designware Multimedia Card Interface Driver406gitlab # [ 0.615603] sdhci-pltfm: SDHCI platform and OF driver helper407gitlab # [ 0.616630] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.00408gitlab # [ 0.617339] hub 1-0:1.0: USB hub found409gitlab # [ 0.617394] hub 1-0:1.0: 6 ports detected410gitlab # [ 0.618240] hid: raw HID events driver (C) Jiri Kosina411gitlab # [ 0.618686] usbcore: registered new interface driver usbhid412gitlab # [ 0.618692] usbhid: USB HID core driver413gitlab # [ 0.647128] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available414gitlab # [ 0.649565] drop_monitor: Initializing network drop monitor service415gitlab # [ 0.649801] NET: Registered PF_INET6 protocol family416gitlab # [ 0.650712] Segment Routing with IPv6417gitlab # [ 0.650730] In-situ OAM (IOAM) with IPv6418gitlab # [ 0.650763] NET: Registered PF_PACKET protocol family419gitlab # [ 0.651012] 9pnet: Installing 9P2000 support420gitlab # [ 0.653504] Key type dns_resolver registered421gitlab # [ 0.663200] registered taskstats version 1422gitlab # [ 0.663547] Loading compiled-in X.509 certificates423gitlab # [ 0.678505] Demotion targets for Node 0: null424gitlab # [ 0.678745] Key type .fscrypt registered425gitlab # [ 0.678752] Key type fscrypt-provisioning registered426gitlab # [ 0.678901] ima: No TPM chip found, activating TPM-bypass!427gitlab # [ 0.678921] ima: Allocated hash algorithm: sha1428gitlab # [ 0.678946] ima: No architecture policies found429gitlab # [ 0.679792] input: gpio-keys as /devices/platform/gpio-keys/input/input0430gitlab # [ 0.702219] clk: Disabling unused clocks431gitlab # [ 0.702245] PM: genpd: Disabling unused power domains432gitlab # [ 0.706038] Freeing unused kernel memory: 4736K433gitlab # [ 0.706241] Run /init as init process434gitlab # [ 0.740625] systemd[1]: Successfully made /usr/ read-only.435gitlab # [ 0.864623] usb 1-1: new high-speed USB device number 2 using ehci-pci436gitlab # [ 1.014513] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1437gitlab # [ 1.074757] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)438gitlab # [ 1.074793] systemd[1]: Detected virtualization qemu.439gitlab # [ 1.074831] systemd[1]: Detected architecture arm64.440gitlab # [ 1.074838] systemd[1]: Running in initrd.441gitlab # [ 1.075275] systemd[1]: Initializing machine ID from random generator.442gitlab # [ 1.075413] systemd[1]: Hostname set to <gitlab>.443gitlab # [ 1.132962] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input0444gitlab # [ 1.252622] usb 1-2: new high-speed USB device number 3 using ehci-pci445gitlab # [ 1.254116] systemd[1]: bpf-restrict-fs: LSM BPF program attached446gitlab # [ 1.316216] systemd[1]: Queued start job for default target Initrd Default Target.447gitlab # [ 1.331478] systemd[1]: Created slice Slice /system/modprobe.448gitlab # [ 1.331724] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.449gitlab # [ 1.331759] systemd[1]: Expecting device /dev/disk/by-label/nix-store...450gitlab # [ 1.331772] systemd[1]: Expecting device /dev/disk/by-label/nixos...451gitlab # [ 1.331796] systemd[1]: Reached target Path Units.452gitlab # [ 1.331811] systemd[1]: Reached target Slice Units.453gitlab # [ 1.331833] systemd[1]: Reached target Swaps.454gitlab # [ 1.331850] systemd[1]: Reached target Timer Units.455gitlab # [ 1.332014] systemd[1]: Listening on D-Bus System Message Bus Socket.456gitlab # [ 1.332178] systemd[1]: Listening on Journal Socket (/dev/log).457gitlab # [ 1.332335] systemd[1]: Listening on Journal Sockets.458gitlab # [ 1.332481] systemd[1]: Listening on udev Control Socket.459gitlab # [ 1.332567] systemd[1]: Listening on udev Kernel Socket.460gitlab # [ 1.332673] systemd[1]: Reached target Socket Units.461gitlab # [ 1.334385] systemd[1]: Starting Create List of Static Device Nodes...462gitlab # [ 1.336045] systemd[1]: Starting Load Kernel Module 9pnet_virtio...463gitlab # [ 1.336158] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs464gitlab # [ 1.337893] systemd[1]: Mounting Kernel Configuration File System...465gitlab # [ 1.354413] systemd[1]: Starting Journal Service...466gitlab # [ 1.356400] systemd[1]: Starting Load Kernel Modules...467gitlab # [ 1.356547] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os468gitlab # [ 1.358416] systemd[1]: Starting Coldplug All udev Devices...469gitlab # [ 1.361955] systemd[1]: Finished Create List of Static Device Nodes.470gitlab # [ 1.363207] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.471gitlab # [ 1.363754] systemd[1]: Finished Load Kernel Module 9pnet_virtio.472gitlab # [ 1.364145] systemd[1]: Mounted Kernel Configuration File System.473gitlab # [ 1.366455] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...474gitlab # [ 1.385118] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.475gitlab # [ 1.387344] systemd[1]: Starting Create Static Device Nodes in /dev...476gitlab # [ 1.389551] systemd-journald[95]: Collecting audit messages is disabled.477gitlab # [ 1.393956] [drm] pci: virtio-gpu-pci detected at 0000:00:08.0478gitlab # [ 1.394191] [drm] features: -virgl +edid -resource_blob -host_visible479gitlab # [ 1.394195] [drm] features: -context_init480gitlab # [ 1.395162] [drm] number of scanouts: 1481gitlab # [ 1.395175] [drm] number of cap sets: 0482gitlab # [ 1.398831] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.483gitlab # [ 1.399036] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev484gitlab # [ 1.399676] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic485gitlab # [ 1.399684] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 0486gitlab # [ 1.402921] Console: switching to colour frame buffer device 160x50487gitlab # [ 1.406984] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device488gitlab # [ 1.408570] systemd[1]: Finished Create Static Device Nodes in /dev.489gitlab # [ 1.408955] systemd[1]: Reached target Preparation for Local File Systems.490gitlab # [ 1.409290] systemd[1]: Reached target Local File Systems.491gitlab # [ 1.413824] systemd[1]: Starting Rule-based Manager for Device Events and Files...492gitlab # [ 1.426763] systemd[1]: Finished Load Kernel Modules.493gitlab # [ 1.429754] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2494gitlab # [ 1.430116] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input0495gitlab # [ 1.440978] systemd[1]: Starting Apply Kernel Variables...496gitlab # [ 1.457443] systemd[1]: Finished Apply Kernel Variables.497gitlab # [ 1.460292] systemd[1]: Started Journal Service.498gitlab # [ 1.459820] systemd-modules-load[96]: Using 4 probe threads499gitlab # [ 1.462685] systemd-modules-load[96]: Module 'virtio_balloon' is built in500gitlab # [ 1.463861] systemd-modules-load[96]: Module 'virtio_console' is built in501gitlab # [ 1.465106] systemd-modules-load[96]: Module 'virtio_rng' is built in502gitlab # [ 1.466612] systemd-modules-load[96]: Inserted module 'dm_mod'503gitlab # [ 1.467663] systemd-modules-load[96]: Inserted module 'virtio_gpu'504gitlab # [ 1.469028] systemd-udevd[105]: Using default interface naming scheme 'v261'.505gitlab # [ 1.470237] systemd[1]: Starting Create System Files and Directories...506gitlab # [ 1.479810] systemd[1]: Finished Create System Files and Directories.507gitlab # [ 1.481011] systemd[1]: Started Rule-based Manager for Device Events and Files.508gitlab # [ 1.500404] systemd[1]: Starting Virtual Console Setup...509gitlab # [ 1.545886] systemd-vconsole-setup[130]: Configuration of first virtual console was skipped, ignoring remaining ones.510gitlab # [ 1.548314] systemd[1]: Finished Virtual Console Setup.511gitlab # [ 1.991953] systemd[1]: Finished Coldplug All udev Devices.512gitlab # [ 1.993429] systemd[1]: Reached target System Initialization.513gitlab # [ 1.994753] systemd[1]: Reached target Basic System.514gitlab # [ 2.117387] (udev-worker)[124]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.515gitlab # [ 2.124546] (udev-worker)[124]: Network interface NamePolicy= disabled on kernel command line.516gitlab # [ 2.130200] (udev-worker)[122]: Network interface NamePolicy= disabled on kernel command line.517gitlab # [ 2.135350] systemd[1]: Found device /dev/disk/by-label/nixos.518gitlab # [ 2.139550] systemd[1]: Found device /dev/disk/by-label/nix-store.519gitlab # [ 2.141279] systemd[1]: Reached target Initrd Root Device.520gitlab # [ 2.142999] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...521gitlab # [ 2.183443] systemd-fsck[152]: nixos: clean, 12/65536 files, 13019/262144 blocks522gitlab # [ 2.187465] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.523gitlab # [ 2.364310] systemd[1]: Mounting /sysroot...524gitlab # [ 2.399218] EXT4-fs (vda): mounted filesystem b03b4e32-2437-44f6-91a3-d20db517cabe r/w with ordered data mode. Quota mode: none.525gitlab # [ 2.402421] systemd[1]: Mounted /sysroot.526gitlab # [ 2.403308] systemd[1]: Reached target Initrd Root File System.527gitlab # [ 2.407779] systemd[1]: Mounting /sysroot/nix/.ro-store...528gitlab # [ 2.410317] systemd[1]: Mounting /sysroot/nix/.rw-store...529gitlab # [ 2.412712] systemd[1]: Mounting /sysroot/run...530gitlab # [ 2.429303] systemd[1]: Mounting /sysroot/tmp/shared...531gitlab # [ 2.439115] systemd[1]: Mounting /sysroot/tmp/xchg...532gitlab # [ 2.443359] systemd[1]: Starting Mountpoints Configured in the Real Root...533gitlab # [ 2.458498] systemd-sysroot-fstab-check[166]: /sysroot should be mounted in the initrd, will request daemon-reload.534gitlab # [ 2.474689] systemd[1]: Mounted /sysroot/nix/.rw-store.535gitlab # [ 2.476831] systemd[1]: Mounted /sysroot/run.536gitlab # [ 2.478915] systemd[1]: Mounted /sysroot/tmp/shared.537gitlab # [ 2.507999] systemd[1]: Starting rw-sysroot-nix-store.service...538gitlab # [ 2.516552] systemd[1]: Mounted /sysroot/tmp/xchg.539gitlab # [ 2.518392] systemd[1]: Reload requested from client PID 166 ('systemd-sysroot') (unit initrd-parse-etc.service)...540gitlab # [ 2.521022] systemd[1]: Reloading...541gitlab # [ 2.598142] erofs (device vdb): mounted with root inode @ nid 128.542gitlab # [ 2.664048] systemd[1]: Reloading finished in 145 ms.543gitlab # [ 2.669453] (udev-worker)[141]: mtd0ro: Failed to find and pin callout binary "/nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/mtd_probe": No such file or directory544gitlab # [ 2.671831] (udev-worker)[141]: mtd0ro: /nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory545gitlab # [ 2.706850] systemd-sysroot-fstab-check[166]: Requesting initrd-fs.target/start/replace...546gitlab # [ 2.709053] systemd[1]: Mounted /sysroot/nix/.ro-store.547gitlab # [ 2.709882] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.548gitlab # [ 2.711016] systemd[1]: Finished rw-sysroot-nix-store.service.549gitlab # [ 2.711887] systemd-sysroot-fstab-check[166]: Requesting swap.target/start/replace...550gitlab # [ 2.716740] systemd[1]: initrd-parse-etc.service: Deactivated successfully.551gitlab # [ 2.717932] systemd[1]: Finished Mountpoints Configured in the Real Root.552gitlab # [ 2.719110] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.553gitlab # [ 2.725165] systemd[1]: Starting rw-sysroot-nix-store.service...554gitlab # [ 2.726243] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.555gitlab # [ 2.727482] systemd[1]: Stopped Virtual Console Setup.556gitlab # [ 2.728401] systemd[1]: Stopping Virtual Console Setup...557gitlab # [ 2.729344] systemd[1]: Starting Virtual Console Setup...558gitlab # [ 2.762544] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.559gitlab # [ 2.764977] systemd[1]: Finished rw-sysroot-nix-store.service.560gitlab # [ 2.774866] systemd-vconsole-setup[196]: Configuration of first virtual console was skipped, ignoring remaining ones.561gitlab # [ 2.777573] systemd[1]: Finished Virtual Console Setup.562gitlab # [ 3.363477] systemd[1]: Mounting /sysroot/nix/store...563gitlab # [ 3.419200] systemd[1]: Mounted /sysroot/nix/store.564gitlab # [ 3.420517] systemd[1]: Reached target Initrd File Systems.565gitlab # [ 3.421783] systemd[1]: Starting Find NixOS closure...566gitlab # [ 3.423207] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...567gitlab # [ 3.454143] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.568gitlab # [ 3.456141] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.569gitlab # [ 3.467748] systemd[1]: Finished Find NixOS closure.570gitlab # [ 3.469257] systemd[1]: Reached target Initrd Default Target.571gitlab # [ 3.470598] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...572gitlab # [ 3.491760] systemd[1]: Stopped target Initrd Default Target.573gitlab # [ 3.493166] systemd[1]: Stopped target Basic System.574gitlab # [ 3.494388] systemd[1]: Stopped target Initrd Root Device.575gitlab # [ 3.495616] systemd[1]: Stopped target Path Units.576gitlab # [ 3.496794] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.577gitlab # [ 3.498486] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.578gitlab # [ 3.500233] systemd[1]: Stopped target Slice Units.579gitlab # [ 3.502329] systemd[1]: Stopped target Socket Units.580gitlab # [ 3.503551] systemd[1]: Stopped target System Initialization.581gitlab # [ 3.504911] systemd[1]: Stopped target Swaps.582gitlab # [ 3.506967] systemd[1]: Stopped target Timer Units.583gitlab # [ 3.508165] systemd[1]: dbus.socket: Deactivated successfully.584gitlab # [ 3.512395] systemd[1]: Closed D-Bus System Message Bus Socket.585gitlab # [ 3.515194] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.586gitlab # [ 3.517512] systemd[1]: Stopped Find NixOS closure.587gitlab # [ 3.519727] systemd[1]: Starting Load Kernel Module 9pnet_virtio...588gitlab # [ 3.521547] systemd[1]: Starting rw-sysroot-nix-store.service...589gitlab # [ 3.523122] systemd[1]: systemd-sysctl.service: Deactivated successfully.590gitlab # [ 3.524741] systemd[1]: Stopped Apply Kernel Variables.591gitlab # [ 3.525978] systemd[1]: systemd-modules-load.service: Deactivated successfully.592gitlab # [ 3.527734] systemd[1]: Stopped Load Kernel Modules.593gitlab # [ 3.529049] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.594gitlab # [ 3.530832] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.595gitlab # [ 3.532610] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.596gitlab # [ 3.534309] systemd[1]: Stopped Create System Files and Directories.597gitlab # [ 3.535716] systemd[1]: Stopped target Local File Systems.598gitlab # [ 3.537294] systemd[1]: Stopped target Preparation for Local File Systems.599gitlab # [ 3.538926] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.600gitlab # [ 3.540649] systemd[1]: Stopped Coldplug All udev Devices.601gitlab # [ 3.541869] systemd[1]: Stopping Rule-based Manager for Device Events and Files...602gitlab # [ 3.543495] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.603gitlab # [ 3.545405] systemd[1]: Stopped Virtual Console Setup.604gitlab # [ 3.546536] systemd[1]: initrd-cleanup.service: Deactivated successfully.605gitlab # [ 3.547953] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.606gitlab # [ 3.549493] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.607gitlab # [ 3.551081] systemd[1]: Finished rw-sysroot-nix-store.service.608gitlab # [ 3.552543] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.609gitlab # [ 3.554108] systemd[1]: Finished Load Kernel Module 9pnet_virtio.610gitlab # [ 3.555355] systemd[1]: systemd-udevd.service: Deactivated successfully.611gitlab # [ 3.556793] systemd[1]: Stopped Rule-based Manager for Device Events and Files.612gitlab # [ 3.558297] systemd[1]: systemd-udevd.service: Consumed 1.995s CPU time over 2.127s wall clock time, 32.1M memory peak.613gitlab # [ 3.560297] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.614gitlab # [ 3.561687] systemd[1]: Closed udev Control Socket.615gitlab # [ 3.562664] systemd[1]: Starting Cleanup udev Database...616gitlab # [ 3.563695] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.617gitlab # [ 3.565188] systemd[1]: Stopped Create Static Device Nodes in /dev.618gitlab # [ 3.566361] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.619gitlab # [ 3.567810] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.620gitlab # [ 3.569159] systemd[1]: kmod-static-nodes.service: Deactivated successfully.621gitlab # [ 3.570398] systemd[1]: Stopped Create List of Static Device Nodes.622gitlab # [ 3.600500] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.623gitlab # [ 3.602200] systemd[1]: Finished Cleanup udev Database.624gitlab # [ 3.603377] systemd[1]: Reached target Switch Root.625gitlab # [ 3.604662] systemd[1]: Starting NixOS Activation...626gitlab # [ 3.714237] initrd-nixos-activation-start[222]: booting system configuration /nix/store/6b0rfvdx2flkyw3l6cvkv2fm5h59ya45-nixos-system-gitlab-test627gitlab # [ 3.750229] initrd-nixos-activation-start[222]: running activation script...628gitlab # [ 3.987293] initrd-nixos-activation-start[245]: setting up /etc...629gitlab # [ 4.104463] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.630gitlab # [ 4.106340] systemd[1]: Finished NixOS Activation.631gitlab # [ 4.107480] systemd[1]: Starting Switch Root...632gitlab # [ 4.140177] systemd[1]: Switching root.633gitlab # [ 4.271320] systemd-journald[95]: Received SIGTERM from PID 1 (systemd).634gitlab # [ 4.852980] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)635gitlab # [ 4.853063] systemd[1]: Detected virtualization qemu.636gitlab # [ 4.853134] systemd[1]: Detected architecture arm64.637gitlab # [ 4.853259] systemd[1]: Detected first boot.638gitlab # [ 4.854087] systemd[1]: Initializing machine ID from random generator.639gitlab # [ 5.058931] systemd[1]: bpf-restrict-fs: LSM BPF program attached640gitlab # [ 5.135022] systemd[1]: Applying preset policy.641gitlab # [ 5.317293] systemd[1]: Populated /etc with preset unit settings.642gitlab # [ 5.462729] systemd[1]: initrd-switch-root.service: Deactivated successfully.643gitlab # [ 5.463573] systemd[1]: Stopped initrd-switch-root.service.644gitlab # [ 5.465718] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.645gitlab # [ 5.469591] systemd[1]: Created slice Slice /system/getty.646gitlab # [ 5.471690] systemd[1]: Created slice GitLab DevOps Platform Slice.647gitlab # [ 5.473909] systemd[1]: Created slice User and Session Slice.648gitlab # [ 5.474484] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.649gitlab # [ 5.475035] systemd[1]: Started Forward Password Requests to Wall Directory Watch.650gitlab # [ 5.475470] systemd[1]: Expecting device /dev/hvc0...651gitlab # [ 5.475984] systemd[1]: Expecting device /dev/ttyAMA0...652gitlab # [ 5.476525] systemd[1]: Reached target Local Encrypted Volumes.653gitlab # [ 5.477679] systemd[1]: Stopped target initrd-fs.target.654gitlab # [ 5.478366] systemd[1]: Stopped target initrd-root-fs.target.655gitlab # [ 5.478771] systemd[1]: Stopped target initrd-switch-root.target.656gitlab # [ 5.479172] systemd[1]: Reached target Virtual Machines and Containers.657gitlab # [ 5.479576] systemd[1]: Reached target Path Units.658gitlab # [ 5.479963] systemd[1]: Reached target Remote File Systems.659gitlab # [ 5.480355] systemd[1]: Reached target Slice Units.660gitlab # [ 5.480920] systemd[1]: Reached target Swaps.661gitlab # [ 5.483310] systemd[1]: Listening on Query the User Interactively for a Password.662gitlab # [ 5.485857] systemd[1]: Listening on Process Core Dump Socket.663gitlab # [ 5.487717] systemd[1]: Listening on Credential Encryption/Decryption.664gitlab # [ 5.489699] systemd[1]: Listening on Factory Reset Management.665gitlab # [ 5.490446] systemd[1]: Listening on Hostname Service Socket.666gitlab # [ 5.495456] systemd[1]: Starting Journal Log Access Socket...667gitlab # [ 5.497000] systemd[1]: Listening on Journal Audit Socket.668gitlab # [ 5.498522] systemd[1]: Listening on Console Output Muting Service Socket.669gitlab # [ 5.499187] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.670gitlab # [ 5.499659] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os671gitlab # [ 5.500193] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki672gitlab # [ 5.502408] systemd[1]: Listening on Disk Repartitioning Service Socket.673gitlab # [ 5.502963] systemd[1]: Listening on udev Control Socket.674gitlab # [ 5.503471] systemd[1]: Listening on udev Varlink Socket.675gitlab # [ 5.508077] systemd[1]: Mounting Huge Pages File System...676gitlab # [ 5.512364] systemd[1]: Mounting POSIX Message Queue File System...677gitlab # [ 5.519638] systemd[1]: Mounting Kernel Debug File System...678gitlab # [ 5.531633] systemd[1]: Mounting Kernel Trace File System...679gitlab # [ 5.535463] systemd[1]: Starting Create List of Static Device Nodes...680gitlab # [ 5.541253] systemd[1]: Starting Load Kernel Module 9pnet_virtio...681gitlab # [ 5.541948] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs682gitlab # [ 5.549450] systemd[1]: Mounting Kernel Configuration File System...683gitlab # [ 5.549852] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm684gitlab # [ 5.550155] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore685gitlab # [ 5.556754] systemd[1]: Starting Load Kernel Module fuse...686gitlab # [ 5.557302] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67687gitlab # [ 5.566444] systemd[1]: Starting Journal Service...688gitlab # [ 5.570065] systemd[1]: Starting Load Kernel Modules...689gitlab # [ 5.576777] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...690gitlab # [ 5.582071] systemd[1]: Starting Remount Root and Kernel File Systems...691gitlab # [ 5.582159] fuse: init (API version 7.45)692gitlab # [ 5.583686] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os693gitlab # [ 5.598766] systemd[1]: Starting Coldplug All udev Devices...694gitlab # [ 5.605714] systemd[1]: Listening on Journal Log Access Socket.695gitlab # [ 5.612698] systemd[1]: Mounted Huge Pages File System.696gitlab # [ 5.613508] systemd[1]: Mounted POSIX Message Queue File System.697gitlab # [ 5.614239] systemd[1]: Mounted Kernel Debug File System.698gitlab # [ 5.614933] systemd[1]: Mounted Kernel Trace File System.699gitlab # [ 5.615947] systemd[1]: Finished Create List of Static Device Nodes.700gitlab # [ 5.618318] systemd-journald[317]: Collecting audit messages is enabled.701gitlab # [ 5.619091] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.702gitlab # [ 5.619838] systemd[1]: Finished Load Kernel Module 9pnet_virtio.703gitlab # [ 5.620395] systemd[1]: Mounted Kernel Configuration File System.704gitlab # [ 5.621117] systemd[1]: modprobe@fuse.service: Deactivated successfully.705gitlab # [ 5.621771] systemd[1]: Finished Load Kernel Module fuse.706gitlab # [ 5.622604] systemd[1]: Finished Load Kernel Modules.707gitlab # [ 5.632805] systemd[1]: Mounting FUSE Control File System...708gitlab # [ 5.635243] systemd[1]: Starting Firewall...709gitlab # [ 5.636791] EXT4-fs (vda): re-mounted b03b4e32-2437-44f6-91a3-d20db517cabe.710gitlab # [ 5.644763] systemd[1]: Starting Apply Kernel Variables...711gitlab # [ 5.660525] systemd[1]: Queued start job for default target Multi-User System.712gitlab # [ 5.662042] systemd[1]: Unnecessary job was removed for /dev/vdb.713gitlab # [ 5.663271] systemd[1]: systemd-journald.service: Deactivated successfully.714gitlab # [ 5.664754] systemd-modules-load[318]: Using 3 probe threads715gitlab # [ 5.669243] systemd-modules-load[318]: Module 'atkbd' is built in716gitlab # [ 5.670608] systemd-modules-load[318]: Module 'loop' is built in717gitlab # [ 5.671882] systemd-modules-load[318]: Inserted module 'tls'[ 5.672240] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...718gitlab # 719gitlab # [ 5.674127] systemd[1]: Started Journal Service.720gitlab # [ 5.676765] systemd-oomd[319]: No swap; memory pressure usage will be degraded721gitlab # [ 5.680448] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.722gitlab # [ 5.681845] systemd[1]: Finished Remount Root and Kernel File Systems.723gitlab # [ 5.687229] systemd[1]: Mounted FUSE Control File System.724gitlab # [ 5.694998] systemd[1]: Finished Apply Kernel Variables.725gitlab # [ 5.696753] systemd[1]: Listening on Disk Image Download Service Socket.726gitlab # [ 5.700955] systemd[1]: Starting Flush Journal to Persistent Storage...727gitlab # [ 5.702074] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore728gitlab # [ 5.713112] systemd[1]: Starting Load/Save OS Random Seed...729gitlab # [ 5.715901] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os730gitlab # [ 5.717438] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.731gitlab # [ 5.722435] systemd[1]: Starting Create Static Device Nodes in /dev...732gitlab # [ 5.724727] systemd-journald[317]: Received client request to flush runtime journal.733gitlab # [ 5.759036] systemd[1]: Finished Load/Save OS Random Seed.734gitlab # [ 5.760157] systemd[1]: Reached target First Boot Complete.735gitlab # [ 5.761200] systemd[1]: Finished Create Static Device Nodes in /dev.736gitlab # [ 5.762198] systemd[1]: Reached target Preparation for Local File Systems.737gitlab # [ 5.763197] systemd[1]: Starting Rule-based Manager for Device Events and Files...738gitlab # [ 5.764397] systemd[1]: Finished Flush Journal to Persistent Storage.739gitlab # [ 5.788516] systemd-udevd[363]: Using default interface naming scheme 'v261'.740gitlab # [ 5.815199] systemd[1]: Started Rule-based Manager for Device Events and Files.741gitlab # [ 5.991849] systemd[1]: Finished Firewall.742gitlab # [ 6.213132] systemd[1]: Finished Coldplug All udev Devices.743gitlab # [ 6.267587] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs744gitlab # [ 6.303218] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.745gitlab # [ 6.334049] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped.746gitlab # [ 6.348440] (udev-worker)[402]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.747gitlab # [ 6.353561] (udev-worker)[402]: Network interface NamePolicy= disabled on kernel command line.748gitlab # [ 6.355742] (udev-worker)[401]: Network interface NamePolicy= disabled on kernel command line.749gitlab # [ 6.374494] systemd[1]: Condition check resulted in Virtio network device being skipped.750gitlab # [ 6.376723] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore751gitlab # [ 6.379162] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67752gitlab # [ 6.382867] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore753gitlab # [ 6.385212] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os754gitlab # [ 6.386995] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os755gitlab # [ 6.441500] mousedev: PS/2 mouse device common for all mice756gitlab # [ 6.465542] systemd[1]: Mounting /run/wrappers...757gitlab # [ 6.490041] systemd[1]: Mounted /run/wrappers.758gitlab # [ 6.492204] systemd[1]: Reached target Local File Systems.759gitlab # [ 6.494870] systemd[1]: Listening on Boot Loader Control Service Socket.760gitlab # [ 6.497159] systemd[1]: Starting register-nix-paths.service...761gitlab # [ 6.500569] systemd[1]: Starting Create SUID/SGID Wrappers...762gitlab # [ 6.501514] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.763gitlab # [ 6.512148] systemd[1]: Starting Save Transient machine-id to Disk...764gitlab # [ 6.514796] systemd[1]: Starting Create System Files and Directories...765gitlab # [ 6.544122] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.766gitlab # [ 6.548827] systemd[1]: Finished Save Transient machine-id to Disk.767gitlab # [ 6.557908] systemd[1]: Finished Create System Files and Directories.768gitlab # [ 6.564276] systemd[1]: Starting Rebuild Journal Catalog...769gitlab # [ 6.584291] systemd[1]: Starting Record System Boot/Shutdown in UTMP...770gitlab # [ 6.603944] systemd[1]: Finished Record System Boot/Shutdown in UTMP.771gitlab # [ 6.610212] systemd[1]: Finished Rebuild Journal Catalog.772gitlab # [ 6.613460] systemd[1]: Starting Update is Completed...773gitlab # [ 6.646457] systemd[1]: Finished Update is Completed.774gitlab # [ 6.674884] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.775gitlab # [ 6.677117] systemd[1]: Finished Create SUID/SGID Wrappers.776gitlab # [ 6.678869] systemd[1]: suid-sgid-wrappers.service: Consumed 133ms CPU time over 175ms wall clock time, 6.4M memory peak, 1.9M read from disk.777gitlab # [ 6.863826] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:09.0/virtio8/input/input3778gitlab # [ 6.951409] systemd[1]: Finished register-nix-paths.service.779gitlab # [ 6.952688] systemd[1]: Reached target System Initialization.780gitlab # [ 6.954505] systemd[1]: Started Discard unused filesystem blocks once a week.781gitlab # [ 6.956213] systemd[1]: Started logrotate.timer.782gitlab # [ 6.957442] systemd[1]: Started Daily Cleanup of Temporary Directories.783gitlab # [ 6.958794] systemd[1]: Reached target Timer Units.784gitlab # [ 6.960080] systemd[1]: Listening on D-Bus System Message Bus Socket.785gitlab # [ 6.961485] systemd[1]: Listening on Nix Daemon Socket.786gitlab # [ 6.971701] systemd[1]: Starting nixbot CI service socket...787gitlab # [ 6.973069] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.788gitlab # [ 6.974370] systemd[1]: Listening on nixbot CI service socket.789gitlab # [ 6.982746] systemd[1]: Reached target Socket Units.790gitlab # [ 6.985042] systemd[1]: Starting D-Bus System Message Bus...791gitlab # [ 6.986627] systemd[1]: Starting Virtual Console Setup...792gitlab # [ 7.016228] dbus-broker-launch[568]: Looking up NSS user entry for 'systemd-timesync'...793gitlab # [ 7.018605] dbus-broker-launch[568]: NSS returned no entry for 'systemd-timesync'794gitlab # [ 7.020116] dbus-broker-launch[568]: Invalid user-name in /nix/store/9zcyylhndlnizbwn48627991zr1jxg4p-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"795gitlab # [ 7.023290] systemd[1]: Started D-Bus System Message Bus.796gitlab # [ 7.027829] systemd-vconsole-setup[569]: Configuration of first virtual console was skipped, ignoring remaining ones.797gitlab # [ 7.031063] dbus-broker-launch[568]: Ready798gitlab # [ 7.032263] systemd[1]: Finished Virtual Console Setup.799gitlab # [ 7.033582] systemd[1]: Reached target Basic System.800gitlab # [ 7.035443] systemd[1]: Started backdoor.service.801gitlab # [ 7.037301] systemd[1]: Starting gitlab-config.service...802gitlab # [ 7.040000] systemd[1]: Starting Import lastlog data into lastlog2 database...803gitlab # [ 7.058229] systemd[1]: Starting Logrotate configuration check...804gitlab # [ 7.065894] systemd[1]: Starting Name Service Cache Daemon (nsncd)...805gitlab # [ 7.071409] systemd[1]: Starting Post-Boot Actions...806gitlab # [ 7.074066] systemd[1]: Started Reset console on configuration changes.807gitlab # [ 7.078996] logrotate[576]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want.808gitlab # [ 7.081369] logrotate[576]: reading config file /nix/store/22afkmbkaxyx50bj9lbkkpmz7pd69hsf-logrotate.conf809gitlab # connecting to host...810gitlab # [ 7.088130] logrotate[576]: note: 'monthly' overrides previously specified 'weekly'811gitlab # [ 7.089208] logrotate[576]: Reading state from file: /var/lib/logrotate.status812gitlab # [ 7.090198] logrotate[576]: state file /var/lib/logrotate.status does not exist813gitlab # [ 7.091297] logrotate[576]: Allocating hash table for state file, size 64 entries814gitlab # [ 7.092942] logrotate[576]: Handling 4 logs815gitlab # [ 7.096354] logrotate[576]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed816gitlab # [ 7.098882] logrotate[576]: considering log /var/log/btmp817gitlab # [ 7.099729] logrotate[576]: Creating new state818gitlab # [ 7.104086] logrotate[576]: Now: 2026-09-25 15:54819gitlab # [ 7.104954] logrotate[576]: Last rotated at 2026-09-25 15:00820gitlab # [ 7.105739] logrotate[576]: log does not need rotating (log has already been rotated)821gitlab: Guest shell says: b'Spawning backdoor root shell...\n'822gitlab # [ 7.106856] logrotate[576]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed823gitlab # [ 7.109098] logrotate[576]: considering log /var/log/wtmp824gitlab: connected to guest root shell825gitlab # [ 7.109823] logrotate[576]: Creating new state826gitlab: (connecting took 7.56 seconds)827gitlab # [ 7.110443] logrotate[576]: Now: 2026-09-25 15:54828gitlab: (finished: waiting for the VM to finish booting, in 7.56 seconds)829gitlab # [ 7.111070] logrotate[576]: Last rotated at 2026-09-25 15:00830gitlab # [ 7.111801] logrotate[576]: log does not need rotating (log has already been rotated)831gitlab # [ 7.113118] logrotate[576]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed832gitlab # [ 7.115035] logrotate[576]: switching euid from 0 to 165 and egid from 0 to 165 (pid 576)833gitlab # [ 7.116826] logrotate[576]: considering log /var/gitlab/state/log/*.log834gitlab # [ 7.117764] logrotate[576]: log /var/gitlab/state/log/*.log does not exist -- skipping835gitlab # [ 7.118815] logrotate[576]: Creating new state836gitlab # [ 7.119473] logrotate[576]: switching euid from 165 to 0 and egid from 165 to 0 (pid 576)837gitlab # [ 7.120536] logrotate[576]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed838gitlab # [ 7.122200] logrotate[576]: switching euid from 0 to 60 and egid from 0 to 60 (pid 576)839gitlab # [ 7.123289] logrotate[576]: considering log /var/log/nginx/*.log840gitlab # [ 7.124167] logrotate[576]: log /var/log/nginx/*.log does not exist -- skipping841gitlab # [ 7.125221] logrotate[576]: Creating new state842gitlab # [ 7.125861] logrotate[576]: not running postrotate script, since no logs were rotated843gitlab # [ 7.126872] logrotate[576]: switching euid from 60 to 0 and egid from 60 to 0 (pid 576)844gitlab # [ 7.128322] nsncd[577]: Sep 25 15:54:43.006 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"845gitlab # [ 7.131280] systemd[1]: Starting resolvconf update...846gitlab # [ 7.132856] systemd[1]: Finished Logrotate configuration check.847gitlab # [ 7.136153] systemd[1]: Finished Post-Boot Actions.848gitlab # [ 7.137148] systemd[1]: Started Name Service Cache Daemon (nsncd).849gitlab # [ 7.138225] systemd[1]: Finished Import lastlog data into lastlog2 database.850gitlab # [ 7.139348] systemd[1]: Reached target Host and Network Name Lookups.851gitlab # [ 7.141791] systemd[1]: Reached target User and Group Name Lookups.852gitlab # [ 7.142885] systemd[1]: Starting User Login Management...853gitlab # [ 7.181805] systemd[1]: Stopped target Host and Network Name Lookups.854gitlab # [ 7.182883] systemd[1]: Stopping Host and Network Name Lookups...855gitlab # [ 7.183841] systemd[1]: Stopped target User and Group Name Lookups.856gitlab # [ 7.184907] systemd[1]: Stopping User and Group Name Lookups...857gitlab # [ 7.185801] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...858gitlab # [ 7.187602] systemd[1]: nscd.service: Deactivated successfully.859gitlab # [ 7.189273] systemd-logind[612]: New seat seat0.860gitlab # [ 7.191306] systemd[1]: Stopped Name Service Cache Daemon (nsncd).861gitlab # [ 7.192679] systemd[1]: nscd.service: Consumed 23ms CPU time over 121ms wall clock time, 7.6M memory peak, 5.8M read from disk.862gitlab # [ 7.194318] systemd-logind[612]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard)863gitlab # [ 7.196254] systemd-logind[612]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard)864gitlab # [ 7.197525] systemd-logind[612]: Watching system buttons on /dev/input/event0 (gpio-keys)865gitlab # [ 7.198665] systemd[1]: Starting Name Service Cache Daemon (nsncd)...866gitlab # [ 7.201827] systemd[1]: Started User Login Management.867gitlab # [ 7.206343] systemd[1]: Starting linger-users.service...868gitlab # [ 7.227015] systemd[1]: linger-users.service: Deactivated successfully.869gitlab # [ 7.228250] systemd[1]: Finished linger-users.service.870gitlab # [ 7.233125] nsncd[666]: Sep 25 15:54:43.123 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"871gitlab # [ 7.235224] systemd[1]: Finished resolvconf update.872gitlab # [ 7.235994] systemd[1]: Started Name Service Cache Daemon (nsncd).873gitlab # [ 7.237040] systemd[1]: Reached target Preparation for Network.874gitlab # [ 7.237885] systemd[1]: Reached target Host and Network Name Lookups.875gitlab # [ 7.238805] systemd[1]: Reached target User and Group Name Lookups.876gitlab # [ 7.239716] systemd[1]: Starting DHCP Client...877gitlab # [ 7.242192] systemd[1]: Starting Address configuration of eth1...878gitlab # [ 7.247212] systemd[1]: Starting Extra networking commands....879gitlab # [ 7.276284] network-addresses-eth1-start[694]: adding address 192.168.1.1/24... done880gitlab # [ 7.281937] network-addresses-eth1-start[694]: adding address 2001:db8:1::1/64... done881gitlab # [ 7.296856] systemd[1]: Finished Address configuration of eth1.882gitlab # [ 7.326748] systemd[1]: Finished Extra networking commands..883gitlab # [ 7.327886] systemd[1]: Reached target Network.884gitlab # [ 7.330033] systemd[1]: Starting gitlab-workhorse.service...885gitlab # [ 7.331950] dhcpcd[702]: dhcpcd-10.3.2 starting886gitlab # [ 7.336152] systemd[1]: Starting Nginx Web Server...887gitlab # [ 7.337266] dhcpcd[760]: dev: loaded udev888gitlab # [ 7.340599] systemd[1]: Starting PostgreSQL Server...889gitlab # [ 7.344728] systemd[1]: Starting Redis Server - redis-gitlab...890gitlab # [ 7.349618] systemd[1]: Starting Permit User Sessions...891gitlab # [ 7.352695] 8021q: 802.1Q VLAN Support v1.8892gitlab # [ 7.353096] 8021q: adding VLAN 0 to HW filter on device eth1893gitlab # [ 7.363142] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.894gitlab # [ 7.380123] systemd[1]: Finished Permit User Sessions.895gitlab # [ 7.386073] systemd[1]: Started Getty on tty1.896gitlab # [ 7.389154] systemd[1]: Reached target Login Prompts.897gitlab # [ 7.393748] cfg80211: Loading compiled-in X.509 certificates for regulatory database898gitlab # [ 7.402749] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'899gitlab # [ 7.403219] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'900gitlab # [ 7.404563] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2901gitlab # [ 7.404927] cfg80211: failed to load regulatory.db902gitlab # [ 7.456985] 8021q: adding VLAN 0 to HW filter on device eth0903gitlab # [ 7.457769] dhcpcd[760]: eth0: waiting for carrier904gitlab # [ 7.463431] dhcpcd[760]: eth0: carrier acquired905gitlab # [ 7.465035] dhcpcd[760]: DUID 00:01:00:01:32:49:55:c3:52:54:00:12:34:56906gitlab # [ 7.466042] dhcpcd[760]: eth0: IAID 00:12:34:56907gitlab # [ 7.466752] dhcpcd[760]: eth0: adding address fe80::5054:ff:fe12:3456908gitlab # [ 7.515313] nginx-pre-start[793]: nginx: the configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf syntax is ok909gitlab # [ 7.517244] nginx-pre-start[793]: nginx: configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf test is successful910gitlab # [ 7.521114] systemd[1]: Started Nginx Web Server.911gitlab # [ 7.535979] redis-server[790]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.912gitlab # [ 7.538461] redis-server[790]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo913gitlab # [ 7.539671] redis-server[790]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=790, just started914gitlab # [ 7.543836] redis-server[790]: Configuration loaded915gitlab # [ 7.545021] systemd[1]: Started Redis Server - redis-gitlab.916gitlab # [ 7.545894] redis[790]: monotonic clock: ARM CNTVCT @ 1000 ticks/us917gitlab # [ 7.547382] redis[790]: Running mode=standalone, port=0.918gitlab # [ 7.548398] redis[790]: Server initialized919gitlab # [ 7.549459] postgresql-pre-start[796]: The files belonging to this database system will be owned by user "postgres".920gitlab # [ 7.551238] postgresql-pre-start[796]: This user must also own the server process.921gitlab # [ 7.552615] redis[790]: Ready to accept connections unix922gitlab # [ 7.553519] postgresql-pre-start[796]: The database cluster will be initialized with locale "en_US.UTF-8".923gitlab # [ 7.554882] postgresql-pre-start[796]: The default database encoding has accordingly been set to "UTF8".924gitlab # [ 7.556304] postgresql-pre-start[796]: The default text search configuration will be set to "english".925gitlab # [ 7.558675] postgresql-pre-start[796]: Data page checksums are enabled.926gitlab # [ 7.559642] postgresql-pre-start[796]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok927gitlab # [ 7.561194] redis[790]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.928gitlab # [ 7.563131] postgresql-pre-start[796]: creating subdirectories ... ok929gitlab # [ 7.564061] postgresql-pre-start[796]: selecting dynamic shared memory implementation ... posix930gitlab # [ 7.597084] postgresql-pre-start[796]: selecting default "max_connections" ... 100931gitlab # [ 7.654054] postgresql-pre-start[796]: selecting default "shared_buffers" ... 128MB932gitlab # [ 7.907924] systemd[1]: Started gitlab-workhorse.service.933gitlab # [ 8.110648] postgresql-pre-start[796]: selecting default time zone ... UTC934gitlab # [ 8.113216] postgresql-pre-start[796]: creating configuration files ... ok935gitlab # [ 8.215303] gitlab-workhorse[821]: time="2026-09-25T15:54:44Z" level=info msg=Starting build_time=19700101.000000 version=19.0.4936gitlab # [ 8.219448] gitlab-workhorse[821]: time="2026-09-25T15:54:44Z" level=info msg="Using redis/go-redis"937gitlab # [ 8.223408] gitlab-workhorse[821]: time="2026-09-25T15:54:44Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix938gitlab # [ 8.378015] postgresql-pre-start[796]: running bootstrap script ... ok939gitlab # [ 8.981214] postgresql-pre-start[796]: performing post-bootstrap initialization ... ok940gitlab # [ 9.168450] postgresql-pre-start[796]: syncing data to disk ... ok941gitlab # [ 9.170142] postgresql-pre-start[796]: initdb: warning: enabling "trust" authentication for local connections942gitlab # [ 9.172351] postgresql-pre-start[796]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.943gitlab # [ 9.175798] postgresql-pre-start[796]: Success. You can now start the database server using:944gitlab # [ 9.177792] postgresql-pre-start[796]: pg_ctl -D /var/lib/postgresql/18 -l logfile start945gitlab # [ 9.299449] postgres[846]: [846] LOG: starting PostgreSQL 18.4 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit946gitlab # [ 9.302599] postgres[846]: [846] LOG: listening on IPv6 address "::1", port 5432947gitlab # [ 9.304360] postgres[846]: [846] LOG: listening on IPv4 address "127.0.0.1", port 5432948gitlab # [ 9.306215] postgres[846]: [846] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"949gitlab # [ 9.311413] postgres[864]: [864] LOG: database system was shut down at 2026-09-25 15:54:44 GMT950gitlab # [ 9.321454] postgres[846]: [846] LOG: database system is ready to accept connections951gitlab # [ 9.325617] systemd[1]: Started PostgreSQL Server.952gitlab # [ 9.330331] systemd[1]: Starting PostgreSQL Setup Scripts...953gitlab # [ 9.388227] dhcpcd[760]: eth0: soliciting a DHCP lease954gitlab # [ 9.408701] dhcpcd[760]: eth0: offered 10.0.2.15 from 10.0.2.2955gitlab # [ 9.424382] dhcpcd[760]: eth0: probing address 10.0.2.15/24956gitlab # [ 9.545875] postgresql-setup-start[875]: CREATE DATABASE957gitlab # [ 9.558705] dhcpcd[760]: eth0: soliciting an IPv6 router958gitlab # [ 9.559697] dhcpcd[760]: eth0: Router Advertisement from fe80::2959gitlab # [ 9.560625] dhcpcd[760]: eth0: adding address fec0::5054:ff:fe12:3456/64960gitlab # [ 9.561615] dhcpcd[760]: eth0: adding route to fec0::/64961gitlab # [ 9.562391] dhcpcd[760]: eth0: adding default route via fe80::2962gitlab # [ 9.577596] postgresql-setup-start[885]: CREATE ROLE963gitlab # [ 9.591149] postgresql-setup-start[887]: ALTER DATABASE964gitlab # [ 9.621136] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[880]: mkdir -p /var/gitlab/state/home/.ssh: OK965gitlab # [ 9.623520] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[880]: chmod 700 /var/gitlab/state/home/.ssh: OK966gitlab # [ 9.625886] postgresql-setup-start[892]: CREATE ROLE967gitlab # [ 9.627141] systemd[1]: Finished PostgreSQL Setup Scripts.968gitlab # [ 9.628622] systemd[1]: Reached target PostgreSQL.969gitlab # [ 9.629745] systemd[1]: Starting gitlab-postgresql.service...970gitlab # [ 9.685016] systemd[1]: Finished gitlab-config.service.971gitlab # [ 9.689129] systemd[1]: Started gitaly.service.972gitlab # [ 9.693517] systemd[1]: Starting gitlab-db-config.service...973gitlab # [ 9.767300] gitlab-postgresql-start[911]: CREATE DATABASE974gitlab # [ 9.828597] gitlab-postgresql-start[921]: CREATE EXTENSION975gitlab # [ 9.916904] gitlab-postgresql-start[923]: CREATE EXTENSION976gitlab # [ 9.918604] systemd[1]: Finished gitlab-postgresql.service.977gitlab # [ 10.359849] gitaly[914]: time="2026-09-25T15:54:46.249Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/0r0qd84zrxhqq373hk9spv8j9pg0qlnq-gitaly.toml' and fix the errors that are printed." pid=914978gitlab # [ 10.372332] gitaly[914]: time="2026-09-25T15:54:46.250Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=914979gitlab # [ 10.375453] gitaly[914]: time="2026-09-25T15:54:46.250Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=914980gitlab # [ 10.378998] gitaly[914]: time="2026-09-25T15:54:46.250Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=914981gitlab # [ 10.382023] gitaly[914]: time="2026-09-25T15:54:46.250Z" level=info msg="Starting Gitaly" pid=914 version=982gitlab # [ 10.576772] gitaly[914]: time="2026-09-25T15:54:46.467Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=216 pid=914983gitlab # [ 10.578920] gitaly[914]: time="2026-09-25T15:54:46.467Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=914984gitlab # [ 10.580972] gitaly[914]: time="2026-09-25T15:54:46.467Z" level=info msg="finished initializing command factory" duration_ms=0 pid=914985gitlab # [ 10.582924] gitaly[914]: time="2026-09-25T15:54:46.467Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-914/git-exec-3992958239.d/git pid=914986gitlab # [ 10.585368] gitaly[914]: time="2026-09-25T15:54:46.472Z" level=info msg="finished detecting git version" duration_ms=4 pid=914987gitlab # [ 10.587252] gitaly[914]: time="2026-09-25T15:54:46.472Z" level=info msg="using Git version" pid=914 version=2.53.g988gitlab # [ 10.589039] gitaly[914]: time="2026-09-25T15:54:46.473Z" level=info msg="clearing disk cache object folder" pid=914 storage=default989gitlab # [ 10.591100] gitaly[914]: time="2026-09-25T15:54:46.473Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1710448810 pid=914 storage=default990gitlab # [ 10.594729] gitaly[914]: time="2026-09-25T15:54:46.473Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1710448810 pid=914 storage=default991gitlab # [ 10.598376] gitaly[914]: time="2026-09-25T15:54:46.473Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=914992gitlab # [ 10.601407] gitaly[914]: time="2026-09-25T15:54:46.473Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=914993gitlab # [ 10.604376] gitaly[914]: time="2026-09-25T15:54:46.473Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1710448810 clear_duration_ms=0 pid=914 storage=default994gitlab # [ 10.607532] gitaly[914]: time="2026-09-25T15:54:46.486Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=914995gitlab # [ 10.610383] gitaly[914]: time="2026-09-25T15:54:46.486Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=914 schema=unix996gitlab # [ 10.612946] gitaly[914]: time="2026-09-25T15:54:46.486Z" level=info msg="listening at address" address=/run/gitaly/gitaly-914/sock.d/intern pid=914 schema=unix997gitlab # [ 10.615248] gitaly[914]: time="2026-09-25T15:54:46.486Z" level=info msg="counting repositories" pid=914998gitlab # [ 10.616924] gitaly[914]: time="2026-09-25T15:54:46.487Z" level=info msg="starting to count repositories" pid=914 storage_path=/var/gitlab/state/repositories999gitlab # [ 10.619205] gitaly[914]: time="2026-09-25T15:54:46.488Z" level=info msg="finished tempdir cleaner walk" pid=914 storage=default time_ms=11000gitlab # [ 10.621299] gitaly[914]: time="2026-09-25T15:54:46.488Z" level=info msg="completed counting repositories" counting_duration_ms=1 pid=914 storage_path=/var/gitlab/state/repositories1001gitlab # [ 10.623973] gitaly[914]: time="2026-09-25T15:54:46.488Z" level=info msg="completed counting all repositories" counting_duration=1 pid=9141002gitlab # [ 10.626200] gitaly[914]: time="2026-09-25T15:54:46.489Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=9141003gitlab # [ 10.628198] gitaly[914]: time="2026-09-25T15:54:46.489Z" level=info msg="Started Gitaly" duration_ms=238 pid=9141004gitlab # [ 10.630911] gitaly[914]: time="2026-09-25T15:54:46.489Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=9141005gitlab # [ 10.632920] gitaly[914]: time="2026-09-25T15:54:46.489Z" level=info msg="maintenance: daily scheduled" pid=914 scheduled="2026-09-26 12:00:00 +0000 UTC"1006gitlab # [ 10.836757] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[917]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1007gitlab # [ 10.839649] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[917]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1008gitlab # [ 10.853295] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[917]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1009gitlab # [ 13.896648] gitaly[914]: time="2026-09-25T15:54:49.786Z" level=info msg="License database preloaded" duration_ms=3536 pid=9141010gitlab # [ 14.240422] dhcpcd[760]: eth0: leased 10.0.2.15 for 86400 seconds1011gitlab # [ 14.241836] dhcpcd[760]: eth0: adding route to 10.0.2.0/241012gitlab # [ 14.243362] dhcpcd[760]: eth0: adding default route via 10.0.2.21013gitlab # [ 14.346167] systemd[1]: Started DHCP Client.1014gitlab # [ 14.347253] systemd[1]: Reached target Network is Online.1015gitlab # [ 54.259281] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[917]: W, [2026-09-25T15:55:30.149288 #917] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1016gitlab # [ 57.097519] postgres[1432]: [1432] ERROR: relation "feature_gates" does not exist at character 5231017gitlab # [ 57.097773] postgres[1432]: [1432] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod),1018gitlab # [ 57.097837] postgres[1432]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1019gitlab # [ 57.097896] postgres[1432]: c.collname, col_description(a.attrelid, a.attnum) AS comment,1020gitlab # [ 57.097953] postgres[1432]: attidentity AS identity,1021gitlab # [ 57.098010] postgres[1432]: attgenerated as attgenerated1022gitlab # [ 57.098069] postgres[1432]: FROM pg_attribute a1023gitlab # [ 57.098140] postgres[1432]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1024gitlab # [ 57.098199] postgres[1432]: LEFT JOIN pg_type t ON a.atttypid = t.oid1025gitlab # [ 57.098257] postgres[1432]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1026gitlab # [ 57.098307] postgres[1432]: WHERE a.attrelid = '"feature_gates"'::regclass1027gitlab # [ 57.098362] postgres[1432]: AND a.attnum > 0 AND NOT a.attisdropped1028gitlab # [ 57.098420] postgres[1432]: ORDER BY a.attnum1029gitlab # [ 57.098475] postgres[1432]:1030gitlab # [ 58.275394] gitlab-workhorse[821]: time="2026-09-25T15:55:34Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1031gitlab # [ 62.265179] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1483]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping1032gitlab # [ 62.267848] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1483]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping1033gitlab # [ 68.281215] gitlab-workhorse[821]: time="2026-09-25T15:55:44Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1034gitlab # [ 78.289313] gitlab-workhorse[821]: time="2026-09-25T15:55:54Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1035gitlab # [ 82.796192] postgres[1577]: [1577] ERROR: relation "table_without_models" does not exist1036gitlab # [ 82.798109] postgres[1577]: [1577] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1037gitlab # [ 88.290023] gitlab-workhorse[821]: time="2026-09-25T15:56:04Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1038gitlab # [ 98.299779] gitlab-workhorse[821]: time="2026-09-25T15:56:14Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1039gitlab # [ 99.698764] systemd[1]: Finished gitlab-db-config.service.1040gitlab # [ 99.704088] systemd[1]: Started gitlab-sidekiq.service.1041gitlab # [ 99.704915] systemd[1]: Starting gitlab.service...1042gitlab # [ 100.246327] bundle[1816]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1043gitlab # [ 100.249862] bundle[1816]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1044gitlab # [ 100.253225] bundle[1816]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1045gitlab # [ 104.835170] sidekiq-cluster[1815]: {"severity":"INFO","time":"2026-09-25T15:56:20.696Z","message":"Starting cluster with 1 processes"}1046gitlab # [ 104.950595] bundle[1816]: {"timestamp":"2026-09-25T15:56:20.811Z","pid":1816,"message":"* Enabling systemd notification integration"}1047gitlab # [ 104.953696] bundle[1816]: {"timestamp":"2026-09-25T15:56:20.840Z","pid":1816,"message":"Puma starting in cluster mode..."}1048gitlab # [ 104.956214] bundle[1816]: {"timestamp":"2026-09-25T15:56:20.840Z","pid":1816,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"}1049gitlab # [ 104.958865] bundle[1816]: {"timestamp":"2026-09-25T15:56:20.840Z","pid":1816,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1050gitlab # [ 104.962147] bundle[1816]: {"timestamp":"2026-09-25T15:56:20.840Z","pid":1816,"message":"* Min threads: 0"}1051gitlab # [ 104.964248] bundle[1816]: {"timestamp":"2026-09-25T15:56:20.840Z","pid":1816,"message":"* Max threads: 4"}1052gitlab # [ 104.966331] bundle[1816]: {"timestamp":"2026-09-25T15:56:20.841Z","pid":1816,"message":"* Environment: production"}1053gitlab # [ 104.968656] bundle[1816]: {"timestamp":"2026-09-25T15:56:20.841Z","pid":1816,"message":"* Master PID: 1816"}1054gitlab # [ 104.970808] bundle[1816]: {"timestamp":"2026-09-25T15:56:20.841Z","pid":1816,"message":"* Workers: 2"}1055gitlab # [ 104.972935] bundle[1816]: {"timestamp":"2026-09-25T15:56:20.841Z","pid":1816,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"}1056gitlab # [ 104.975594] bundle[1816]: {"timestamp":"2026-09-25T15:56:20.841Z","pid":1816,"message":"* Preloading application"}1057gitlab # [ 108.309433] gitlab-workhorse[821]: time="2026-09-25T15:56:24Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1058gitlab # [ 118.317346] gitlab-workhorse[821]: time="2026-09-25T15:56:34Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1059gitlab # [ 128.320995] gitlab-workhorse[821]: time="2026-09-25T15:56:44Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1060gitlab # [ 138.329406] gitlab-workhorse[821]: time="2026-09-25T15:56:54Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1061gitlab # [ 143.345245] bundle[1816]: W, [2026-09-25T15:56:59.235313 #1816] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1062gitlab # [ 146.937195] sidekiq-cluster[1864]: 2026-09-25T15:57:02.827Z pid=1864 tid=wrk INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>1864, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>426213376}1063gitlab # [ 147.429231] sidekiq-cluster[1864]: W, [2026-09-25T15:57:03.319834 #1864] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1064gitlab # [ 147.499759] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:03.390Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1065gitlab # [ 148.338683] gitlab-workhorse[821]: time="2026-09-25T15:57:04Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1066gitlab # [ 150.672322] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.562Z","message":"GitLab reliable fetch activated!"}1067gitlab # [ 150.672565] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.562Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1068gitlab # [ 150.703648] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.594Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1069gitlab # [ 150.707476] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.596Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"}1070gitlab # [ 150.711364] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.599Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1071gitlab # [ 150.715010] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.602Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1072gitlab # [ 150.719087] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.604Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1073gitlab # [ 150.722848] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.606Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1074gitlab # [ 150.726666] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.609Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1075gitlab # [ 150.731127] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.611Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1076gitlab # [ 150.734711] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.612Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1077gitlab # [ 150.738259] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.614Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1078gitlab # [ 150.742133] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.616Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1079gitlab # [ 150.745712] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.618Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1080gitlab # [ 150.749189] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.620Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1081gitlab # [ 150.752702] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.622Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1082gitlab # [ 150.756142] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.624Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1083gitlab # [ 150.759559] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.626Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1084gitlab # [ 150.762876] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.629Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1085gitlab # [ 150.765910] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.632Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1086gitlab # [ 150.769102] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.634Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1087gitlab # [ 150.771960] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.637Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1088gitlab # [ 150.775306] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.639Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1089gitlab # [ 150.778619] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.641Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1090gitlab # [ 150.781495] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.644Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1091gitlab # [ 150.784210] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.646Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1092gitlab # [ 150.786866] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.648Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1093gitlab # [ 150.789453] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.651Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1094gitlab # [ 150.791792] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.653Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1095gitlab # [ 150.794328] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.656Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1096gitlab # [ 150.796767] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.658Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1097gitlab # [ 150.799142] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.661Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1098gitlab # [ 150.801430] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.663Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1099gitlab # [ 150.803867] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.666Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1100gitlab # [ 150.806280] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.669Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1101gitlab # [ 150.808791] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.671Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1102gitlab # [ 150.811119] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.673Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1103gitlab # [ 150.813589] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.675Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1104gitlab # [ 150.815927] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.677Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1105gitlab # [ 150.818211] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.680Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1106gitlab # [ 150.820334] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.682Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1107gitlab # [ 150.822417] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.684Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1108gitlab # [ 150.824706] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.685Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1109gitlab # [ 150.826850] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.687Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1110gitlab # [ 150.829144] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.689Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1111gitlab # [ 150.831389] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.692Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1112gitlab # [ 150.833761] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.694Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1113gitlab # [ 150.836241] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.696Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1114gitlab # [ 150.838600] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.698Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1115gitlab # [ 150.841006] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.701Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1116gitlab # [ 150.843287] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.703Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1117gitlab # [ 150.845614] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.705Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1118gitlab # [ 150.847758] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.708Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1119gitlab # [ 150.850031] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.711Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1120gitlab # [ 150.852187] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.724Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1121gitlab # [ 150.854576] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.727Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1122gitlab # [ 150.856866] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.730Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1123gitlab # [ 150.859060] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.733Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1124gitlab # [ 150.861368] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.735Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1125gitlab # [ 150.917592] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.806Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1126gitlab # [ 150.919899] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.810Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1127gitlab # [ 150.922407] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.812Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1128gitlab # [ 150.924515] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.815Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1129gitlab # [ 150.927296] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.818Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1130gitlab # [ 150.929736] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.820Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1131gitlab # [ 150.933020] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.823Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1132gitlab # [ 150.935463] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.825Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1133gitlab # [ 150.939785] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.829Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1134gitlab # [ 150.943000] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.832Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1135gitlab # [ 150.946155] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.834Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1136gitlab # [ 150.949467] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.835Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1137gitlab # [ 150.952892] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.838Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1138gitlab # [ 150.956169] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.840Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1139gitlab # [ 150.959427] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.842Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1140gitlab # [ 150.962801] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.844Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1141gitlab # [ 150.966089] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.846Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1142gitlab # [ 150.969149] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.848Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1143gitlab # [ 150.972071] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.850Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1144gitlab # [ 150.974799] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.853Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1145gitlab # [ 150.977852] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.855Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1146gitlab # [ 150.980424] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.857Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1147gitlab # [ 150.982834] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.859Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1148gitlab # [ 150.985527] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.861Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1149gitlab # [ 150.988085] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.863Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1150gitlab # [ 150.990509] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.866Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1151gitlab # [ 150.992910] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.868Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1152gitlab # [ 150.995369] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.870Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1153gitlab # [ 150.997837] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.872Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1154gitlab # [ 151.000238] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.875Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1155gitlab # [ 151.002549] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.877Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1156gitlab # [ 151.005001] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.879Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1157gitlab # [ 151.007306] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.880Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1158gitlab # [ 151.009770] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.882Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1159gitlab # [ 151.012067] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.884Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1160gitlab # [ 151.014451] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.886Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1161gitlab # [ 151.016950] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.888Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1162gitlab # [ 151.018943] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.890Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1163gitlab # [ 151.021384] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.895Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1164gitlab # [ 151.023664] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.897Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1165gitlab # [ 151.026094] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.898Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1166gitlab # [ 151.028431] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.910Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1167gitlab # [ 151.030755] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.913Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1168gitlab # [ 151.033071] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.915Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1169gitlab # [ 151.035366] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.918Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1170gitlab # [ 151.037634] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.921Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1171gitlab # [ 151.039855] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.924Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1172gitlab # [ 151.042417] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.926Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1173gitlab # [ 151.044876] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.929Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1174gitlab # [ 151.047053] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.930Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1175gitlab # [ 151.049493] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.934Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1176gitlab # [ 151.051757] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:06.935Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1177gitlab # [ 158.345386] gitlab-workhorse[821]: time="2026-09-25T15:57:14Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1178gitlab # [ 168.346227] gitlab-workhorse[821]: time="2026-09-25T15:57:24Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1179gitlab # [ 178.353452] gitlab-workhorse[821]: time="2026-09-25T15:57:34Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1180gitlab # [ 186.196491] postgres[2593]: [2593] ERROR: relation "table_without_models" does not exist1181gitlab # [ 186.196741] postgres[2593]: [2593] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1182gitlab # [ 188.363120] gitlab-workhorse[821]: time="2026-09-25T15:57:44Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1183gitlab # [ 190.977246] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:46.867Z","message":"Booted Rails 7.2.3.1 application in production environment"}1184gitlab # [ 190.980806] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:46.871Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1185gitlab # [ 190.982879] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:46.871Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1186gitlab # [ 190.984745] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:46.871Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1187gitlab # [ 190.986763] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:46.876Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1188gitlab # [ 191.276322] postgres[2640]: [2640] ERROR: relation "table_without_models" does not exist1189gitlab # [ 191.278494] postgres[2640]: [2640] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1190gitlab # [ 193.679468] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:49.569Z","message":"Cleaning working queues"}1191gitlab # [ 193.680988] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:49.571Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"fb8cd40c0897205a7024ca1f","created_at":"2026-09-25T15:56:15.009Z","trace_propagation_headers":{"sentry-trace":"b132ee2171c54324977c494a3f4287db-763677eba1174ecb","baggage":"sentry-trace_id=b132ee2171c54324977c494a3f4287db,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"ba8cbca09a846a60558722f2beef954a","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-25T15:56:15.020Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-fb8cd40c0897205a7024ca1f: start","job_status":"start","queue_duration_s":94.550902,"scheduling_latency_s":94.550966}1192gitlab # [ 193.901496] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:57:49.791Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"fb8cd40c0897205a7024ca1f","created_at":"2026-09-25T15:56:15.009Z","trace_propagation_headers":{"sentry-trace":"b132ee2171c54324977c494a3f4287db-763677eba1174ecb","baggage":"sentry-trace_id=b132ee2171c54324977c494a3f4287db,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"ba8cbca09a846a60558722f2beef954a","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-25T15:56:15.020Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-fb8cd40c0897205a7024ca1f: done: 0.220519 sec","job_status":"done","queue_duration_s":94.550902,"scheduling_latency_s":94.550966,"redis_calls":29,"redis_duration_s":0.006816,"redis_read_bytes":282,"redis_write_bytes":3632,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.004028,"redis_feature_flag_read_bytes":272,"redis_feature_flag_write_bytes":2385,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001671,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001117,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":15,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.1072,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.11366032000000001,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.220519,"completed_at":"2026-09-25T15:57:49.791Z","load_balancing_strategy":"primary","db_duration_s":0.061473,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1193gitlab # [ 198.325758] bundle[1816]: {"timestamp":"2026-09-25T15:57:54.215Z","pid":1816,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1194gitlab # [ 198.325968] bundle[1816]: {"timestamp":"2026-09-25T15:57:54.215Z","pid":1816,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1195gitlab # [ 198.326072] bundle[1816]: {"timestamp":"2026-09-25T15:57:54.216Z","pid":1816,"message":"! #\u003cThread:0x0000fec1a408e0c0 /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1196gitlab # [ 198.326137] bundle[1816]: {"timestamp":"2026-09-25T15:57:54.216Z","pid":1816,"message":"! #\u003cThread:0x0000fec1a4241f20 /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1197gitlab # [ 198.326218] bundle[1816]: {"timestamp":"2026-09-25T15:57:54.216Z","pid":1816,"message":"Use Ctrl-C to stop"}1198gitlab # [ 198.326279] bundle[1816]: {"timestamp":"2026-09-25T15:57:54.216Z","pid":1816,"message":"Sending status to systemd every 1.0 sec"}1199gitlab # [ 201.662283] bundle[1816]: {"timestamp":"2026-09-25T15:57:57.551Z","pid":1816,"message":"- Worker 1 (PID: 2744) booted in 3.28s, phase: 0"}1200gitlab # [ 201.662547] bundle[1816]: {"timestamp":"2026-09-25T15:57:57.552Z","pid":1816,"message":"- Worker 0 (PID: 2740) booted in 3.3s, phase: 0"}1201gitlab # [ 201.667277] systemd[1]: Started gitlab.service.1202gitlab # [ 201.670539] systemd[1]: Reached target Common target for all GitLab services..1203gitlab # [ 201.672890] systemd[1]: Starting setup-gitlab.service...1204gitlab # [ 201.698119] setup-gitlab-start[2797]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1205gitlab # [ 202.564115] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:15:57:58 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 8421206gitlab # [ 202.567526] setup-gitlab-start[2797]: + TOKEN=nixbot-test-token-of-20-chars1207gitlab # [ 202.569310] setup-gitlab-start[2797]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n'1208gitlab # [ 202.579408] runuser[2813]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1209gitlab # [ 203.120295] setup-gitlab-start[2814]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1210gitlab # [ 203.123673] setup-gitlab-start[2814]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1211gitlab # [ 203.127094] setup-gitlab-start[2814]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1212gitlab # [ 203.954517] setup-gitlab-start[2814]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1213gitlab # [ 203.958044] setup-gitlab-start[2814]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1214gitlab # [ 203.961502] setup-gitlab-start[2814]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1215gitlab # [ 206.711511] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:02.600Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"ec63784c6ae03c865199b87a","created_at":"2026-09-25T15:58:02.531Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"852f0c655466c2fb1fdcb14df853d607","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:02.598Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-ec63784c6ae03c865199b87a: start","job_status":"start","queue_duration_s":0.002239,"scheduling_latency_s":0.002284}1216gitlab # [ 206.826101] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:02.715Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"ec63784c6ae03c865199b87a","created_at":"2026-09-25T15:58:02.531Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"852f0c655466c2fb1fdcb14df853d607","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:02.598Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-ec63784c6ae03c865199b87a: done: 0.115299 sec","job_status":"done","queue_duration_s":0.002239,"scheduling_latency_s":0.002284,"redis_calls":12,"redis_duration_s":0.01586,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.002078,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002853,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.009641,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001288,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024233,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00206946,"duration_s":0.115299,"completed_at":"2026-09-25T15:58:02.715Z","load_balancing_strategy":"primary","db_duration_s":0.009534,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1217gitlab # [ 206.886365] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:02.759Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"472e372c46e6eb1fa623c0a7","created_at":"2026-09-25T15:58:02.630Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8a3b8fa431f1b1daa60c543f3f0d7d4c","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:02.639Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-472e372c46e6eb1fa623c0a7: start","job_status":"start","queue_duration_s":0.120285,"scheduling_latency_s":0.120316}1218gitlab # [ 206.993844] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:02.883Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"472e372c46e6eb1fa623c0a7","created_at":"2026-09-25T15:58:02.630Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8a3b8fa431f1b1daa60c543f3f0d7d4c","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:02.639Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-472e372c46e6eb1fa623c0a7: done: 0.124517 sec","job_status":"done","queue_duration_s":0.120285,"scheduling_latency_s":0.120316,"redis_calls":10,"redis_duration_s":0.036321,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003962,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.008876,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.023483,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023995,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00109102,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.124517,"completed_at":"2026-09-25T15:58:02.883Z","load_balancing_strategy":"primary","db_duration_s":0.054279,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1219gitlab # [ 207.057315] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:02.902Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"6416c550b0c3465d588d93f2","created_at":"2026-09-25T15:58:02.644Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:02.663Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-6416c550b0c3465d588d93f2: start","job_status":"start","queue_duration_s":0.238978,"scheduling_latency_s":0.239005}1220gitlab # [ 207.433914] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:03.324Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"6416c550b0c3465d588d93f2","created_at":"2026-09-25T15:58:02.644Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:02.663Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-6416c550b0c3465d588d93f2: done: 0.421621 sec","job_status":"done","queue_duration_s":0.238978,"scheduling_latency_s":0.239005,"redis_calls":71,"redis_duration_s":0.217757,"redis_read_bytes":120,"redis_write_bytes":42674,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.041455,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000968,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.162417,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.012917,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.145639,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0004152,"duration_s":0.421621,"completed_at":"2026-09-25T15:58:03.323Z","load_balancing_strategy":"primary","db_duration_s":0.014047,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1221gitlab # [ 207.500848] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:03.325Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"4486a1cacc3fb34ff1bb467b","created_at":"2026-09-25T15:58:02.667Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8b8231c3d3830927977ed3eafe8e2a82","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:02.673Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-4486a1cacc3fb34ff1bb467b: start","job_status":"start","queue_duration_s":0.651939,"scheduling_latency_s":0.651959}1222gitlab # [ 207.519176] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:03.367Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"4486a1cacc3fb34ff1bb467b","created_at":"2026-09-25T15:58:02.667Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8b8231c3d3830927977ed3eafe8e2a82","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:02.673Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-4486a1cacc3fb34ff1bb467b: done: 0.041724 sec","job_status":"done","queue_duration_s":0.651939,"scheduling_latency_s":0.651959,"redis_calls":19,"redis_duration_s":0.003544,"redis_read_bytes":22,"redis_write_bytes":4785,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000835,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000339,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.001761,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000609,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028547,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002734,"duration_s":0.041724,"completed_at":"2026-09-25T15:58:03.367Z","load_balancing_strategy":"primary","db_duration_s":0.002948,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1223gitlab # [ 207.569810] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:03.368Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"7f3f67c9d0510d2384368463","created_at":"2026-09-25T15:58:02.678Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"aeb14ad6baff6904841e391957e49f71","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:02.691Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-7f3f67c9d0510d2384368463: start","job_status":"start","queue_duration_s":0.677059,"scheduling_latency_s":0.677078}1224gitlab # [ 207.584850] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:03.379Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1225gitlab # [ 207.588293] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:03.387Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"7f3f67c9d0510d2384368463","created_at":"2026-09-25T15:58:02.678Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"aeb14ad6baff6904841e391957e49f71","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:02.691Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-7f3f67c9d0510d2384368463: done: 0.018626 sec","job_status":"done","queue_duration_s":0.677059,"scheduling_latency_s":0.677078,"redis_calls":10,"redis_duration_s":0.001777,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000389,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001032,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000356,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010429,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002637,"duration_s":0.018626,"completed_at":"2026-09-25T15:58:03.387Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00055,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1226gitlab # [ 207.631557] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:03.388Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"7246ec48ba12582318a3f774","created_at":"2026-09-25T15:58:02.700Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7ae291c4d8305cc2943ce31f3ad2cc54","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:02.713Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-7246ec48ba12582318a3f774: start","job_status":"start","queue_duration_s":0.675335,"scheduling_latency_s":0.675354}1227gitlab # [ 207.646496] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:03.399Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1228gitlab # [ 207.650234] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:03.405Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"7246ec48ba12582318a3f774","created_at":"2026-09-25T15:58:02.700Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7ae291c4d8305cc2943ce31f3ad2cc54","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:02.713Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-7246ec48ba12582318a3f774: done: 0.017259 sec","job_status":"done","queue_duration_s":0.675335,"scheduling_latency_s":0.675354,"redis_calls":10,"redis_duration_s":0.001706,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000353,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000996,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000357,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009838,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00024658,"duration_s":0.017259,"completed_at":"2026-09-25T15:58:03.405Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000417,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1229gitlab # [ 207.693769] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:03.407Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"c02e65afe0559892212ee83b","created_at":"2026-09-25T15:58:02.723Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6ccc0783d8112314fccb90c8f3124022","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:02.729Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-c02e65afe0559892212ee83b: start","job_status":"start","queue_duration_s":0.678163,"scheduling_latency_s":0.678182}1230gitlab # [ 207.708820] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:03.418Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1231gitlab # [ 207.712941] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:03.425Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"c02e65afe0559892212ee83b","created_at":"2026-09-25T15:58:02.723Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6ccc0783d8112314fccb90c8f3124022","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:02.729Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-c02e65afe0559892212ee83b: done: 0.018297 sec","job_status":"done","queue_duration_s":0.678163,"scheduling_latency_s":0.678182,"redis_calls":10,"redis_duration_s":0.001688,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000353,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00097,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000365,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010554,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002454,"duration_s":0.018297,"completed_at":"2026-09-25T15:58:03.425Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000449,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1232gitlab # [ 207.756980] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:03.426Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"3fada1e9f217eefa93dc16bc","created_at":"2026-09-25T15:58:02.738Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cba14fce440317acbccc8302aa8b70e8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:02.744Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-3fada1e9f217eefa93dc16bc: start","job_status":"start","queue_duration_s":0.682632,"scheduling_latency_s":0.682659}1233gitlab # [ 207.771707] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:03.439Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1234gitlab # [ 207.775151] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:03.446Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"3fada1e9f217eefa93dc16bc","created_at":"2026-09-25T15:58:02.738Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cba14fce440317acbccc8302aa8b70e8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:02.744Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-3fada1e9f217eefa93dc16bc: done: 0.019677 sec","job_status":"done","queue_duration_s":0.682632,"scheduling_latency_s":0.682659,"redis_calls":10,"redis_duration_s":0.001765,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000406,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001006,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000353,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011328,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025132,"duration_s":0.019677,"completed_at":"2026-09-25T15:58:03.446Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000456,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1235gitlab # [ 207.818631] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:03.449Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"ae92f45d9ea38c0c9348ca87","created_at":"2026-09-25T15:58:02.749Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a0f5554303d2e685149eeaf1d4be1f4a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:02.755Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-ae92f45d9ea38c0c9348ca87: start","job_status":"start","queue_duration_s":0.694212,"scheduling_latency_s":0.694229}1236gitlab # [ 207.833785] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:03.535Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"ae92f45d9ea38c0c9348ca87","created_at":"2026-09-25T15:58:02.749Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a0f5554303d2e685149eeaf1d4be1f4a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:02.755Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-ae92f45d9ea38c0c9348ca87: done: 0.086143 sec","job_status":"done","queue_duration_s":0.694212,"scheduling_latency_s":0.694229,"redis_calls":12,"redis_duration_s":0.002451,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000367,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001038,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001046,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.053885,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00074922,"exclusive_lock_hold_duration_s":0.050772679999999994,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.086143,"completed_at":"2026-09-25T15:58:03.535Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.023343,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1237gitlab # [ 207.881496] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:03.536Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"b59e7b5aa53c51a806e440fa","created_at":"2026-09-25T15:58:02.775Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3358e18daaef8ac8d6599497eb37fc44","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:02.790Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-b59e7b5aa53c51a806e440fa: start","job_status":"start","queue_duration_s":0.746355,"scheduling_latency_s":0.746373}1238gitlab # [ 207.896621] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:03.574Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"b59e7b5aa53c51a806e440fa","created_at":"2026-09-25T15:58:02.775Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3358e18daaef8ac8d6599497eb37fc44","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:02.790Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-b59e7b5aa53c51a806e440fa: done: 0.038157 sec","job_status":"done","queue_duration_s":0.746355,"scheduling_latency_s":0.746373,"redis_calls":16,"redis_duration_s":0.004788000000000001,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.00334,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001069,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000379,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022619,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00024362,"duration_s":0.038157,"completed_at":"2026-09-25T15:58:03.574Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005482,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1239gitlab # [ 207.941281] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:03.576Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"6f6a37fe69f28a639498f79f","created_at":"2026-09-25T15:58:02.809Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"aad035c066db0b3a1958c6636db5b9d0","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:02.819Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-6f6a37fe69f28a639498f79f: start","job_status":"start","queue_duration_s":0.756657,"scheduling_latency_s":0.756674}1240gitlab # [ 207.956230] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:03.608Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"6f6a37fe69f28a639498f79f","created_at":"2026-09-25T15:58:02.809Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"aad035c066db0b3a1958c6636db5b9d0","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:02.819Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-6f6a37fe69f28a639498f79f: done: 0.032077 sec","job_status":"done","queue_duration_s":0.756657,"scheduling_latency_s":0.756674,"redis_calls":13,"redis_duration_s":0.0023209999999999997,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000386,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001143,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000792,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020962,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00983342,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.032077,"completed_at":"2026-09-25T15:58:03.608Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002632,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1241gitlab # [ 208.002313] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:03.609Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Gitlab::JiraImport::StuckJiraImportJobsWorker","jid":"7827ed721fbd782ea4295302","created_at":"2026-09-25T15:58:02.829Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7d7ce916512b9c741485fd9150a295f3","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b9cf6c9b6c2f8fad80e08693d59af787bb65ad6ffe24ec5d0e1a669517566127","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:02.837Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Gitlab::JiraImport::StuckJiraImportJobsWorker JID-7827ed721fbd782ea4295302: start","job_status":"start","queue_duration_s":0.772567,"scheduling_latency_s":0.772583}1242gitlab # [ 208.016331] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:03.638Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Gitlab::JiraImport::StuckJiraImportJobsWorker","jid":"7827ed721fbd782ea4295302","created_at":"2026-09-25T15:58:02.829Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7d7ce916512b9c741485fd9150a295f3","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b9cf6c9b6c2f8fad80e08693d59af787bb65ad6ffe24ec5d0e1a669517566127","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:02.837Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Gitlab::JiraImport::StuckJiraImportJobsWorker JID-7827ed721fbd782ea4295302: done: 0.028533 sec","job_status":"done","queue_duration_s":0.772567,"scheduling_latency_s":0.772583,"redis_calls":10,"redis_duration_s":0.001759,"redis_read_bytes":9,"redis_write_bytes":1187,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000376,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001034,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000349,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016754,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025236,"duration_s":0.028533,"completed_at":"2026-09-25T15:58:03.638Z","load_balancing_strategy":"primary","db_duration_s":0.004002,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1243gitlab # [ 208.058592] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:03.639Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"872fb564d5e6a514003966e5","created_at":"2026-09-25T15:58:02.842Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"695cd416cf912d4b1e811ef3a97599e8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:02.851Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-872fb564d5e6a514003966e5: start","job_status":"start","queue_duration_s":0.787961,"scheduling_latency_s":0.787977}1244gitlab # [ 208.073665] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:03.663Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"872fb564d5e6a514003966e5","created_at":"2026-09-25T15:58:02.842Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"695cd416cf912d4b1e811ef3a97599e8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:02.851Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-872fb564d5e6a514003966e5: done: 0.023681 sec","job_status":"done","queue_duration_s":0.787961,"scheduling_latency_s":0.787977,"redis_calls":12,"redis_duration_s":0.001978,"redis_read_bytes":11,"redis_write_bytes":1566,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00068,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000941,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000357,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013468,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025166,"duration_s":0.023681,"completed_at":"2026-09-25T15:58:03.663Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000764,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1245gitlab # [ 208.116839] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:03.664Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"5d901bf4aa7eefee5491601b","created_at":"2026-09-25T15:58:02.872Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6772e9944e8977b85fe5793dc2559239","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:02.885Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-5d901bf4aa7eefee5491601b: start","job_status":"start","queue_duration_s":0.779043,"scheduling_latency_s":0.779059}1246gitlab # [ 208.132410] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:03.690Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"5d901bf4aa7eefee5491601b","created_at":"2026-09-25T15:58:02.872Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6772e9944e8977b85fe5793dc2559239","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:02.885Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-5d901bf4aa7eefee5491601b: done: 0.026055 sec","job_status":"done","queue_duration_s":0.779043,"scheduling_latency_s":0.779059,"redis_calls":12,"redis_duration_s":0.002166,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000343,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000972,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000851,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014698,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00049996,"exclusive_lock_hold_duration_s":0.0048734600000000005,"duration_s":0.026055,"completed_at":"2026-09-25T15:58:03.690Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002567,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1247gitlab # [ 208.175795] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:03.692Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"2a7ad865db45e9fce5cb383f","created_at":"2026-09-25T15:58:02.897Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2bdddb98d7d0c719446390a07f546a90","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:02.905Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-2a7ad865db45e9fce5cb383f: start","job_status":"start","queue_duration_s":0.787018,"scheduling_latency_s":0.787034}1248gitlab # [ 208.190874] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:03.721Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"2a7ad865db45e9fce5cb383f","created_at":"2026-09-25T15:58:02.897Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2bdddb98d7d0c719446390a07f546a90","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:02.905Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-2a7ad865db45e9fce5cb383f: done: 0.029453 sec","job_status":"done","queue_duration_s":0.787018,"scheduling_latency_s":0.787034,"redis_calls":10,"redis_duration_s":0.001824,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000389,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001052,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000383,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017349,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025364,"duration_s":0.029453,"completed_at":"2026-09-25T15:58:03.721Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003419,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1249gitlab # [ 208.234032] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:03.723Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"6ef8507d01008fdcf4a2cb50","created_at":"2026-09-25T15:58:02.915Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6f3d304d6c05bc3d6a9a0c68b0616f61","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-25T15:58:02.922Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-6ef8507d01008fdcf4a2cb50: start","job_status":"start","queue_duration_s":0.80124,"scheduling_latency_s":0.801256}1250gitlab # [ 208.247386] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:03.760Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"6ef8507d01008fdcf4a2cb50","created_at":"2026-09-25T15:58:02.915Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6f3d304d6c05bc3d6a9a0c68b0616f61","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-25T15:58:02.922Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-6ef8507d01008fdcf4a2cb50: done: 0.036966 sec","job_status":"done","queue_duration_s":0.80124,"scheduling_latency_s":0.801256,"redis_calls":10,"redis_duration_s":0.002162,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000398,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00117,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000594,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022673,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00036746,"duration_s":0.036966,"completed_at":"2026-09-25T15:58:03.760Z","load_balancing_strategy":"primary","db_duration_s":0.004371,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1251gitlab # [ 208.289305] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:03.762Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"ab7afed7d9df1c0ad8216e74","created_at":"2026-09-25T15:58:02.930Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"be0e17a7b2c4f2c2c85c7f856535cc12","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-25T15:58:02.936Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-ab7afed7d9df1c0ad8216e74: start","job_status":"start","queue_duration_s":0.825614,"scheduling_latency_s":0.825641}1252gitlab # [ 208.302868] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:03.779Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1253gitlab # [ 208.306398] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:03.788Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"ab7afed7d9df1c0ad8216e74","created_at":"2026-09-25T15:58:02.930Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"be0e17a7b2c4f2c2c85c7f856535cc12","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-25T15:58:02.936Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-ab7afed7d9df1c0ad8216e74: done: 0.02655 sec","job_status":"done","queue_duration_s":0.825614,"scheduling_latency_s":0.825641,"redis_calls":10,"redis_duration_s":0.0019299999999999999,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000506,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00105,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000374,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014648,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025656,"duration_s":0.02655,"completed_at":"2026-09-25T15:58:03.788Z","load_balancing_strategy":"primary","db_duration_s":0.00055,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1254gitlab # [ 208.349446] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:03.790Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"0231a58a89cdbd2a6f024d4d","created_at":"2026-09-25T15:58:02.939Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c2db691558664f40e0704b3e5c8a2f5e","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:02.949Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-0231a58a89cdbd2a6f024d4d: start","job_status":"start","queue_duration_s":0.840533,"scheduling_latency_s":0.840549}1255gitlab # [ 208.363912] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:03.818Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"0231a58a89cdbd2a6f024d4d","created_at":"2026-09-25T15:58:02.939Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c2db691558664f40e0704b3e5c8a2f5e","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:02.949Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-0231a58a89cdbd2a6f024d4d: done: 0.028766 sec","job_status":"done","queue_duration_s":0.840533,"scheduling_latency_s":0.840549,"redis_calls":10,"redis_duration_s":0.001656,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00043,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000905,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000321,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016118,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00022532,"duration_s":0.028766,"completed_at":"2026-09-25T15:58:03.818Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002817,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1256gitlab # [ 208.406996] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:03.820Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"096f68160ed3fda5caea500e","created_at":"2026-09-25T15:58:02.957Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b7efa68ad2391dcbc48c53e6787b6ddb","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:02.966Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-096f68160ed3fda5caea500e: start","job_status":"start","queue_duration_s":0.853722,"scheduling_latency_s":0.85374}1257gitlab # [ 208.421821] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:03.849Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"096f68160ed3fda5caea500e","created_at":"2026-09-25T15:58:02.957Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b7efa68ad2391dcbc48c53e6787b6ddb","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:02.966Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-096f68160ed3fda5caea500e: done: 0.029455 sec","job_status":"done","queue_duration_s":0.853722,"scheduling_latency_s":0.85374,"redis_calls":12,"redis_duration_s":0.002309,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000296,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000914,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001099,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015012,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00047326,"exclusive_lock_hold_duration_s":0.00374106,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.029455,"completed_at":"2026-09-25T15:58:03.849Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005294,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1258gitlab # [ 208.467257] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:03.851Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"be9628db897a291f607c323c","created_at":"2026-09-25T15:58:02.969Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c7fe0cbca46e1f7d59451cdcc2624ef7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:02.979Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-be9628db897a291f607c323c: start","job_status":"start","queue_duration_s":0.871523,"scheduling_latency_s":0.87154}1259gitlab # [ 208.482516] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.067Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"be9628db897a291f607c323c","created_at":"2026-09-25T15:58:02.969Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c7fe0cbca46e1f7d59451cdcc2624ef7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:02.979Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-be9628db897a291f607c323c: done: 0.216325 sec","job_status":"done","queue_duration_s":0.871523,"scheduling_latency_s":0.87154,"redis_calls":14,"redis_duration_s":0.001895,"redis_read_bytes":13,"redis_write_bytes":1645,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000548,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000724,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000623,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.186311,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00045676,"exclusive_lock_hold_duration_s":0.19493104,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.216325,"completed_at":"2026-09-25T15:58:04.067Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.037204,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1260gitlab # [ 208.528534] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.068Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"c40499fb6052f631922b5667","created_at":"2026-09-25T15:58:02.987Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e728ca6483b2e9305c9261cf7b84babf","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:02.995Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-c40499fb6052f631922b5667: start","job_status":"start","queue_duration_s":1.072851,"scheduling_latency_s":1.07287}1261gitlab # [ 208.544378] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.416Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"c40499fb6052f631922b5667","created_at":"2026-09-25T15:58:02.987Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e728ca6483b2e9305c9261cf7b84babf","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:02.995Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-c40499fb6052f631922b5667: done: 0.347746 sec","job_status":"done","queue_duration_s":1.072851,"scheduling_latency_s":1.07287,"redis_calls":565,"redis_duration_s":0.058407,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000308,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000801,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.057298,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.300273,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00027898,"duration_s":0.347746,"completed_at":"2026-09-25T15:58:04.416Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000383,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1262gitlab # [ 208.587478] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.420Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"e4855ced4a3d0128961f85b6","created_at":"2026-09-25T15:58:03.000Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3978244c9ba58e045bc1e1e8b489d69f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.009Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-e4855ced4a3d0128961f85b6: start","job_status":"start","queue_duration_s":1.411064,"scheduling_latency_s":1.411085}1263gitlab # [ 208.602461] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.442Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"e4855ced4a3d0128961f85b6","created_at":"2026-09-25T15:58:03.000Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3978244c9ba58e045bc1e1e8b489d69f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.009Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-e4855ced4a3d0128961f85b6: done: 0.022771 sec","job_status":"done","queue_duration_s":1.411064,"scheduling_latency_s":1.411085,"redis_calls":10,"redis_duration_s":0.001389,"redis_read_bytes":9,"redis_write_bytes":1215,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000306,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000793,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00029,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011708,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00022032,"duration_s":0.022771,"completed_at":"2026-09-25T15:58:04.442Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000569,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1264gitlab # [ 208.645647] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.444Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"d9fdfaa0f5d3125d6944b407","created_at":"2026-09-25T15:58:03.040Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bc987fc85d3fce901a5b514fa667cd93","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.057Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-d9fdfaa0f5d3125d6944b407: start","job_status":"start","queue_duration_s":1.386476,"scheduling_latency_s":1.386496}1265gitlab # [ 208.659534] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.469Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"d9fdfaa0f5d3125d6944b407","created_at":"2026-09-25T15:58:03.040Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bc987fc85d3fce901a5b514fa667cd93","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.057Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-d9fdfaa0f5d3125d6944b407: done: 0.025077 sec","job_status":"done","queue_duration_s":1.386476,"scheduling_latency_s":1.386496,"redis_calls":12,"redis_duration_s":0.001618,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000261,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000737,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00062,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015915,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00046958,"exclusive_lock_hold_duration_s":0.00686,"duration_s":0.025077,"completed_at":"2026-09-25T15:58:04.469Z","load_balancing_strategy":"primary","db_duration_s":0.002606,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1266gitlab # [ 208.703037] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.470Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"60f4a91db27ea245398a1505","created_at":"2026-09-25T15:58:03.079Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f3aa1a3f914577341778df5f33f0a55e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-25T15:58:03.091Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-60f4a91db27ea245398a1505: start","job_status":"start","queue_duration_s":1.379116,"scheduling_latency_s":1.379133}1267gitlab # [ 208.717074] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.481Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1268gitlab # [ 208.720881] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.489Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"60f4a91db27ea245398a1505","created_at":"2026-09-25T15:58:03.079Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f3aa1a3f914577341778df5f33f0a55e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-25T15:58:03.091Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-60f4a91db27ea245398a1505: done: 0.019054 sec","job_status":"done","queue_duration_s":1.379116,"scheduling_latency_s":1.379133,"redis_calls":10,"redis_duration_s":0.001363,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000302,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000765,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000296,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011042,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00029858,"duration_s":0.019054,"completed_at":"2026-09-25T15:58:04.489Z","load_balancing_strategy":"primary","db_duration_s":0.000373,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1269gitlab # [ 208.764088] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.490Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"9049c3c86eae3f0be06798d0","created_at":"2026-09-25T15:58:03.102Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bbdf75ce08e5592f48d2f6475b711f59","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.115Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-9049c3c86eae3f0be06798d0: start","job_status":"start","queue_duration_s":1.375477,"scheduling_latency_s":1.375494}1270gitlab # [ 208.778808] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.536Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"9049c3c86eae3f0be06798d0","created_at":"2026-09-25T15:58:03.102Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bbdf75ce08e5592f48d2f6475b711f59","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.115Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-9049c3c86eae3f0be06798d0: done: 0.045953 sec","job_status":"done","queue_duration_s":1.375477,"scheduling_latency_s":1.375494,"redis_calls":14,"redis_duration_s":0.001894,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000531,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00075,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000613,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.014,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024644,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00045596,"exclusive_lock_hold_duration_s":0.02518452,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.045953,"completed_at":"2026-09-25T15:58:04.536Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015634,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1271gitlab # [ 208.824564] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.537Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85bcf6c1ce1023b4b1df4f7c","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85bcf6c1ce1023b4b1df4f7c: start","job_status":"start","queue_duration_s":1.223674,"scheduling_latency_s":1.223692}1272gitlab # [ 208.837463] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.559Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85bcf6c1ce1023b4b1df4f7c","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85bcf6c1ce1023b4b1df4f7c: done: 0.021979 sec","job_status":"done","queue_duration_s":1.223674,"scheduling_latency_s":1.223692,"redis_calls":16,"redis_duration_s":0.002363,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000266,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000787,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00131,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013398,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002751,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021979,"completed_at":"2026-09-25T15:58:04.559Z","load_balancing_strategy":"primary","db_duration_s":0.000355,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1273gitlab # [ 208.879450] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.561Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"076c51de44cb5cdcaa8992f2","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-076c51de44cb5cdcaa8992f2: start","job_status":"start","queue_duration_s":1.246798,"scheduling_latency_s":1.246816}1274gitlab # [ 208.892274] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.576Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"076c51de44cb5cdcaa8992f2","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-076c51de44cb5cdcaa8992f2: done: 0.015326 sec","job_status":"done","queue_duration_s":1.246798,"scheduling_latency_s":1.246816,"redis_calls":13,"redis_duration_s":0.001758,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000547,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001211,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010105,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015326,"completed_at":"2026-09-25T15:58:04.576Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1275gitlab # [ 208.932318] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.577Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"62c1572e5c8722021c765585","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-62c1572e5c8722021c765585: start","job_status":"start","queue_duration_s":1.263447,"scheduling_latency_s":1.263465}1276gitlab # [ 208.945725] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.592Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"62c1572e5c8722021c765585","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-62c1572e5c8722021c765585: done: 0.015117 sec","job_status":"done","queue_duration_s":1.263447,"scheduling_latency_s":1.263465,"redis_calls":13,"redis_duration_s":0.001518,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000979,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010251,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015117,"completed_at":"2026-09-25T15:58:04.592Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1277gitlab # [ 208.986248] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.594Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8e6bd85f4a04e76d115f4912","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8e6bd85f4a04e76d115f4912: start","job_status":"start","queue_duration_s":1.279903,"scheduling_latency_s":1.279919}1278gitlab # [ 208.998855] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.608Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8e6bd85f4a04e76d115f4912","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8e6bd85f4a04e76d115f4912: done: 0.014477 sec","job_status":"done","queue_duration_s":1.279903,"scheduling_latency_s":1.279919,"redis_calls":13,"redis_duration_s":0.0014759999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000552,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000924,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010147,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014477,"completed_at":"2026-09-25T15:58:04.608Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1279gitlab # [ 209.040036] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.609Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"604e008f3509a7b5bfe547e4","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-604e008f3509a7b5bfe547e4: start","job_status":"start","queue_duration_s":1.295647,"scheduling_latency_s":1.295664}1280gitlab # [ 209.052659] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.624Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"604e008f3509a7b5bfe547e4","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-604e008f3509a7b5bfe547e4: done: 0.014681 sec","job_status":"done","queue_duration_s":1.295647,"scheduling_latency_s":1.295664,"redis_calls":13,"redis_duration_s":0.001507,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000974,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010342,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014681,"completed_at":"2026-09-25T15:58:04.624Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1281gitlab # [ 209.092946] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.625Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d456f14a40e94e5e848b6927","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d456f14a40e94e5e848b6927: start","job_status":"start","queue_duration_s":1.311607,"scheduling_latency_s":1.311624}1282gitlab # [ 209.106034] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.640Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d456f14a40e94e5e848b6927","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d456f14a40e94e5e848b6927: done: 0.014533 sec","job_status":"done","queue_duration_s":1.311607,"scheduling_latency_s":1.311624,"redis_calls":13,"redis_duration_s":0.001414,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000891,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010179,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014533,"completed_at":"2026-09-25T15:58:04.640Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1283gitlab # [ 209.147429] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.641Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4d77a025893a2aa635c1384f","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4d77a025893a2aa635c1384f: start","job_status":"start","queue_duration_s":1.32744,"scheduling_latency_s":1.327457}1284gitlab # [ 209.160172] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.656Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4d77a025893a2aa635c1384f","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4d77a025893a2aa635c1384f: done: 0.014427 sec","job_status":"done","queue_duration_s":1.32744,"scheduling_latency_s":1.327457,"redis_calls":13,"redis_duration_s":0.0015999999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000704,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000896,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010039,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014427,"completed_at":"2026-09-25T15:58:04.656Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1285gitlab # [ 209.200549] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.657Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bfa767b20e2dceccfa9ad205","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bfa767b20e2dceccfa9ad205: start","job_status":"start","queue_duration_s":1.343009,"scheduling_latency_s":1.343026}1286gitlab # [ 209.213152] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.671Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bfa767b20e2dceccfa9ad205","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bfa767b20e2dceccfa9ad205: done: 0.014678 sec","job_status":"done","queue_duration_s":1.343009,"scheduling_latency_s":1.343026,"redis_calls":13,"redis_duration_s":0.001485,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000954,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010138,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014678,"completed_at":"2026-09-25T15:58:04.671Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1287gitlab # [ 209.253142] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.673Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1e1491676e2b940643ed4a21","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1e1491676e2b940643ed4a21: start","job_status":"start","queue_duration_s":1.358822,"scheduling_latency_s":1.358839}1288gitlab # [ 209.265803] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.687Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1e1491676e2b940643ed4a21","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1e1491676e2b940643ed4a21: done: 0.014904 sec","job_status":"done","queue_duration_s":1.358822,"scheduling_latency_s":1.358839,"redis_calls":13,"redis_duration_s":0.001595,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000513,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001082,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009939,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014904,"completed_at":"2026-09-25T15:58:04.687Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1289gitlab # [ 209.305716] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.689Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"814fd76607ecd21950df2c42","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-814fd76607ecd21950df2c42: start","job_status":"start","queue_duration_s":1.37487,"scheduling_latency_s":1.374886}1290gitlab # [ 209.318346] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.703Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"814fd76607ecd21950df2c42","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-814fd76607ecd21950df2c42: done: 0.014542 sec","job_status":"done","queue_duration_s":1.37487,"scheduling_latency_s":1.374886,"redis_calls":13,"redis_duration_s":0.001521,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000587,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000934,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009853,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014542,"completed_at":"2026-09-25T15:58:04.703Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1291gitlab # [ 209.360074] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.704Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e45b3ffff0ed64790f462045","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e45b3ffff0ed64790f462045: start","job_status":"start","queue_duration_s":1.390581,"scheduling_latency_s":1.390599}1292gitlab # [ 209.372671] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.719Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e45b3ffff0ed64790f462045","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e45b3ffff0ed64790f462045: done: 0.014808 sec","job_status":"done","queue_duration_s":1.390581,"scheduling_latency_s":1.390599,"redis_calls":13,"redis_duration_s":0.001464,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000522,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000942,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010069,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014808,"completed_at":"2026-09-25T15:58:04.719Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1293gitlab # [ 209.412941] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.720Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f15097e668efa6be784aaf46","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f15097e668efa6be784aaf46: start","job_status":"start","queue_duration_s":1.40655,"scheduling_latency_s":1.406568}1294gitlab # [ 209.426063] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.735Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f15097e668efa6be784aaf46","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f15097e668efa6be784aaf46: done: 0.014544 sec","job_status":"done","queue_duration_s":1.40655,"scheduling_latency_s":1.406568,"redis_calls":13,"redis_duration_s":0.001457,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000924,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010003,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014544,"completed_at":"2026-09-25T15:58:04.735Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1295gitlab # [ 209.467209] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.736Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4693d827c3361f09f354b5f7","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4693d827c3361f09f354b5f7: start","job_status":"start","queue_duration_s":1.422347,"scheduling_latency_s":1.422364}1296gitlab # [ 209.480045] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.751Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4693d827c3361f09f354b5f7","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4693d827c3361f09f354b5f7: done: 0.014488 sec","job_status":"done","queue_duration_s":1.422347,"scheduling_latency_s":1.422364,"redis_calls":13,"redis_duration_s":0.001465,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00052,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000945,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009965,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014488,"completed_at":"2026-09-25T15:58:04.751Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1297gitlab # [ 209.519989] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.752Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d1c565bfe86c8fbc10d1c0e5","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d1c565bfe86c8fbc10d1c0e5: start","job_status":"start","queue_duration_s":1.438134,"scheduling_latency_s":1.438151}1298gitlab # [ 209.532469] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.766Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d1c565bfe86c8fbc10d1c0e5","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d1c565bfe86c8fbc10d1c0e5: done: 0.014443 sec","job_status":"done","queue_duration_s":1.438134,"scheduling_latency_s":1.438151,"redis_calls":13,"redis_duration_s":0.0014789999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000551,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000928,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010275,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014443,"completed_at":"2026-09-25T15:58:04.766Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1299gitlab # [ 209.573464] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.768Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4ede8f932a08e4a9ee71fb74","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4ede8f932a08e4a9ee71fb74: start","job_status":"start","queue_duration_s":1.454033,"scheduling_latency_s":1.45405}1300gitlab # [ 209.586780] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.783Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4ede8f932a08e4a9ee71fb74","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4ede8f932a08e4a9ee71fb74: done: 0.014879 sec","job_status":"done","queue_duration_s":1.454033,"scheduling_latency_s":1.45405,"redis_calls":13,"redis_duration_s":0.001467,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000529,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000938,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010439,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014879,"completed_at":"2026-09-25T15:58:04.783Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1301gitlab # [ 209.626726] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.784Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d0827dd5702633e484b2bdd","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d0827dd5702633e484b2bdd: start","job_status":"start","queue_duration_s":1.470178,"scheduling_latency_s":1.470196}1302gitlab # [ 209.639426] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.798Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d0827dd5702633e484b2bdd","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d0827dd5702633e484b2bdd: done: 0.014316 sec","job_status":"done","queue_duration_s":1.470178,"scheduling_latency_s":1.470196,"redis_calls":13,"redis_duration_s":0.001414,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000513,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000901,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010036,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014316,"completed_at":"2026-09-25T15:58:04.798Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1303gitlab # [ 209.680268] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.800Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d7ae6578b780bd38b21db049","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d7ae6578b780bd38b21db049: start","job_status":"start","queue_duration_s":1.485814,"scheduling_latency_s":1.485831}1304gitlab # [ 209.692815] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.814Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d7ae6578b780bd38b21db049","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d7ae6578b780bd38b21db049: done: 0.014413 sec","job_status":"done","queue_duration_s":1.485814,"scheduling_latency_s":1.485831,"redis_calls":13,"redis_duration_s":0.0014260000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000519,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000907,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010277,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014413,"completed_at":"2026-09-25T15:58:04.814Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1305gitlab # [ 209.733159] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.815Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5a2ddde471a9eb5e460033d4","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5a2ddde471a9eb5e460033d4: start","job_status":"start","queue_duration_s":1.501493,"scheduling_latency_s":1.501511}1306gitlab # [ 209.746723] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.830Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5a2ddde471a9eb5e460033d4","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5a2ddde471a9eb5e460033d4: done: 0.014852 sec","job_status":"done","queue_duration_s":1.501493,"scheduling_latency_s":1.501511,"redis_calls":13,"redis_duration_s":0.0016710000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001121,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010388,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014852,"completed_at":"2026-09-25T15:58:04.830Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1307gitlab # [ 209.788163] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.831Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"91dc22bfff301bee3bfc35e0","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-91dc22bfff301bee3bfc35e0: start","job_status":"start","queue_duration_s":1.517579,"scheduling_latency_s":1.517595}1308gitlab # [ 209.800597] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.846Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"91dc22bfff301bee3bfc35e0","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-91dc22bfff301bee3bfc35e0: done: 0.014579 sec","job_status":"done","queue_duration_s":1.517579,"scheduling_latency_s":1.517595,"redis_calls":13,"redis_duration_s":0.001634,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000511,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001123,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010181,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014579,"completed_at":"2026-09-25T15:58:04.846Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1309gitlab # [ 209.840700] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.847Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dfc755e564fa2830a35ad294","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dfc755e564fa2830a35ad294: start","job_status":"start","queue_duration_s":1.533381,"scheduling_latency_s":1.533398}1310gitlab # [ 209.860526] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.862Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dfc755e564fa2830a35ad294","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dfc755e564fa2830a35ad294: done: 0.01486 sec","job_status":"done","queue_duration_s":1.533381,"scheduling_latency_s":1.533398,"redis_calls":13,"redis_duration_s":0.0018110000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000692,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001119,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010055,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01486,"completed_at":"2026-09-25T15:58:04.862Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1311gitlab # [ 209.919284] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.863Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1a56220a918bcd4c6f400a63","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1a56220a918bcd4c6f400a63: start","job_status":"start","queue_duration_s":1.549483,"scheduling_latency_s":1.5495}1312gitlab # [ 209.933611] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.879Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1a56220a918bcd4c6f400a63","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1a56220a918bcd4c6f400a63: done: 0.016136 sec","job_status":"done","queue_duration_s":1.549483,"scheduling_latency_s":1.5495,"redis_calls":13,"redis_duration_s":0.0017259999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000743,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000983,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010949,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016136,"completed_at":"2026-09-25T15:58:04.879Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1313gitlab # [ 209.973967] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.881Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0256e83a311faa4a6931b55b","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0256e83a311faa4a6931b55b: start","job_status":"start","queue_duration_s":1.566748,"scheduling_latency_s":1.566765}1314gitlab # [ 209.987597] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.895Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0256e83a311faa4a6931b55b","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0256e83a311faa4a6931b55b: done: 0.014938 sec","job_status":"done","queue_duration_s":1.566748,"scheduling_latency_s":1.566765,"redis_calls":13,"redis_duration_s":0.0015379999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000565,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000973,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010332,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014938,"completed_at":"2026-09-25T15:58:04.895Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1315gitlab # [ 210.027946] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.897Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"750449675f62fcfedebfa823","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-750449675f62fcfedebfa823: start","job_status":"start","queue_duration_s":1.58294,"scheduling_latency_s":1.582956}1316gitlab # [ 210.040876] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.911Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"750449675f62fcfedebfa823","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-750449675f62fcfedebfa823: done: 0.014376 sec","job_status":"done","queue_duration_s":1.58294,"scheduling_latency_s":1.582956,"redis_calls":13,"redis_duration_s":0.00143,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000546,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000884,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009826,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014376,"completed_at":"2026-09-25T15:58:04.911Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1317gitlab # [ 210.081378] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.912Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0ec095b03eaae5692632dd5b","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0ec095b03eaae5692632dd5b: start","job_status":"start","queue_duration_s":1.598658,"scheduling_latency_s":1.598677}1318gitlab # [ 210.094704] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.927Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0ec095b03eaae5692632dd5b","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0ec095b03eaae5692632dd5b: done: 0.014839 sec","job_status":"done","queue_duration_s":1.598658,"scheduling_latency_s":1.598677,"redis_calls":13,"redis_duration_s":0.001483,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000548,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000935,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01047,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014839,"completed_at":"2026-09-25T15:58:04.927Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1319gitlab # [ 210.135502] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.929Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d6dcd0c64cd8f09c8d99ab7","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d6dcd0c64cd8f09c8d99ab7: start","job_status":"start","queue_duration_s":1.614758,"scheduling_latency_s":1.614775}1320gitlab # [ 210.148311] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.943Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d6dcd0c64cd8f09c8d99ab7","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d6dcd0c64cd8f09c8d99ab7: done: 0.014626 sec","job_status":"done","queue_duration_s":1.614758,"scheduling_latency_s":1.614775,"redis_calls":13,"redis_duration_s":0.001465,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000542,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000923,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010199,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014626,"completed_at":"2026-09-25T15:58:04.943Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1321gitlab # [ 210.189642] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.944Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a063406faec16fc2c8e23c08","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a063406faec16fc2c8e23c08: start","job_status":"start","queue_duration_s":1.630697,"scheduling_latency_s":1.630714}1322gitlab # [ 210.202463] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.959Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a063406faec16fc2c8e23c08","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a063406faec16fc2c8e23c08: done: 0.014584 sec","job_status":"done","queue_duration_s":1.630697,"scheduling_latency_s":1.630714,"redis_calls":13,"redis_duration_s":0.001462,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000549,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000913,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010123,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014584,"completed_at":"2026-09-25T15:58:04.959Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1323gitlab # [ 210.243350] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.960Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"10bdf8c74a86ba074e03d07f","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-10bdf8c74a86ba074e03d07f: start","job_status":"start","queue_duration_s":1.646592,"scheduling_latency_s":1.646609}1324gitlab # [ 210.256186] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.975Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"10bdf8c74a86ba074e03d07f","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-10bdf8c74a86ba074e03d07f: done: 0.0146 sec","job_status":"done","queue_duration_s":1.646592,"scheduling_latency_s":1.646609,"redis_calls":13,"redis_duration_s":0.0014320000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000908,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010241,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0146,"completed_at":"2026-09-25T15:58:04.975Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1325gitlab # [ 210.296233] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.976Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"79884dd80deaab68ba53b9dc","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-79884dd80deaab68ba53b9dc: start","job_status":"start","queue_duration_s":1.662542,"scheduling_latency_s":1.66256}1326gitlab # [ 210.308661] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.991Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"79884dd80deaab68ba53b9dc","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-79884dd80deaab68ba53b9dc: done: 0.014687 sec","job_status":"done","queue_duration_s":1.662542,"scheduling_latency_s":1.66256,"redis_calls":13,"redis_duration_s":0.00148,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000512,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000968,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01009,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014687,"completed_at":"2026-09-25T15:58:04.991Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1327gitlab # [ 210.349669] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:04.992Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"47a7970590c26bfd5eb0e0bd","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-47a7970590c26bfd5eb0e0bd: start","job_status":"start","queue_duration_s":1.678373,"scheduling_latency_s":1.678389}1328gitlab # [ 210.363085] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.007Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"47a7970590c26bfd5eb0e0bd","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-47a7970590c26bfd5eb0e0bd: done: 0.014757 sec","job_status":"done","queue_duration_s":1.678373,"scheduling_latency_s":1.678389,"redis_calls":13,"redis_duration_s":0.0015379999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000522,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001016,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010153,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014757,"completed_at":"2026-09-25T15:58:05.007Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1329gitlab # [ 210.404722] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.008Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04e0891723d96abd3cb9d3f7","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04e0891723d96abd3cb9d3f7: start","job_status":"start","queue_duration_s":1.694231,"scheduling_latency_s":1.694248}1330gitlab # [ 210.417801] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.023Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04e0891723d96abd3cb9d3f7","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04e0891723d96abd3cb9d3f7: done: 0.014579 sec","job_status":"done","queue_duration_s":1.694231,"scheduling_latency_s":1.694248,"redis_calls":13,"redis_duration_s":0.001448,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000535,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000913,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009899,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014579,"completed_at":"2026-09-25T15:58:05.022Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1331gitlab # [ 210.457926] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.024Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3136bfc6db511bae222a12be","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3136bfc6db511bae222a12be: start","job_status":"start","queue_duration_s":1.709979,"scheduling_latency_s":1.709997}1332gitlab # [ 210.470634] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.038Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3136bfc6db511bae222a12be","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3136bfc6db511bae222a12be: done: 0.014577 sec","job_status":"done","queue_duration_s":1.709979,"scheduling_latency_s":1.709997,"redis_calls":13,"redis_duration_s":0.001446,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000518,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000928,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010002,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014577,"completed_at":"2026-09-25T15:58:05.038Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1333gitlab # [ 210.510879] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.039Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a285c11639bce8da0e7a5e96","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a285c11639bce8da0e7a5e96: start","job_status":"start","queue_duration_s":1.725722,"scheduling_latency_s":1.72574}1334gitlab # [ 210.524399] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.054Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a285c11639bce8da0e7a5e96","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a285c11639bce8da0e7a5e96: done: 0.014382 sec","job_status":"done","queue_duration_s":1.725722,"scheduling_latency_s":1.72574,"redis_calls":13,"redis_duration_s":0.0014550000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000916,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009989,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014382,"completed_at":"2026-09-25T15:58:05.054Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1335gitlab # [ 210.565441] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.055Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"70cb1f2c7128f48d732e4937","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-70cb1f2c7128f48d732e4937: start","job_status":"start","queue_duration_s":1.741464,"scheduling_latency_s":1.741483}1336gitlab # [ 210.578702] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.070Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"70cb1f2c7128f48d732e4937","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-70cb1f2c7128f48d732e4937: done: 0.014522 sec","job_status":"done","queue_duration_s":1.741464,"scheduling_latency_s":1.741483,"redis_calls":13,"redis_duration_s":0.0014650000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00054,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000925,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01005,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014522,"completed_at":"2026-09-25T15:58:05.070Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1337gitlab # [ 210.619739] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.071Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6d025b949e1df69fc592707","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6d025b949e1df69fc592707: start","job_status":"start","queue_duration_s":1.757322,"scheduling_latency_s":1.757339}1338gitlab # [ 210.633152] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.086Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6d025b949e1df69fc592707","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6d025b949e1df69fc592707: done: 0.014627 sec","job_status":"done","queue_duration_s":1.757322,"scheduling_latency_s":1.757339,"redis_calls":13,"redis_duration_s":0.001624,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000676,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000948,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010252,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014627,"completed_at":"2026-09-25T15:58:05.086Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1339gitlab # [ 210.673903] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.087Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7bdd20b04ae725043e2fbea5","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7bdd20b04ae725043e2fbea5: start","job_status":"start","queue_duration_s":1.773348,"scheduling_latency_s":1.773365}1340gitlab # [ 210.687033] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.102Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7bdd20b04ae725043e2fbea5","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7bdd20b04ae725043e2fbea5: done: 0.014452 sec","job_status":"done","queue_duration_s":1.773348,"scheduling_latency_s":1.773365,"redis_calls":13,"redis_duration_s":0.001575,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000672,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000903,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010152,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014452,"completed_at":"2026-09-25T15:58:05.101Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1341gitlab # [ 210.727052] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.103Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9956675c3710916eafc5b8c2","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9956675c3710916eafc5b8c2: start","job_status":"start","queue_duration_s":1.789258,"scheduling_latency_s":1.789275}1342gitlab # [ 210.741067] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.118Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9956675c3710916eafc5b8c2","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9956675c3710916eafc5b8c2: done: 0.014529 sec","job_status":"done","queue_duration_s":1.789258,"scheduling_latency_s":1.789275,"redis_calls":13,"redis_duration_s":0.001597,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000516,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001081,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010262,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014529,"completed_at":"2026-09-25T15:58:05.117Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1343gitlab # [ 210.781677] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.119Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f02023d827905850127810a6","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f02023d827905850127810a6: start","job_status":"start","queue_duration_s":1.805184,"scheduling_latency_s":1.805202}1344gitlab # [ 210.795154] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.133Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f02023d827905850127810a6","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f02023d827905850127810a6: done: 0.0143 sec","job_status":"done","queue_duration_s":1.805184,"scheduling_latency_s":1.805202,"redis_calls":13,"redis_duration_s":0.001431,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000514,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000917,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01013,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0143,"completed_at":"2026-09-25T15:58:05.133Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1345gitlab # [ 210.836596] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.134Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b3e173530ca53e1dff837bb2","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b3e173530ca53e1dff837bb2: start","job_status":"start","queue_duration_s":1.820607,"scheduling_latency_s":1.820625}1346gitlab # [ 210.849525] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.149Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b3e173530ca53e1dff837bb2","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b3e173530ca53e1dff837bb2: done: 0.014789 sec","job_status":"done","queue_duration_s":1.820607,"scheduling_latency_s":1.820625,"redis_calls":13,"redis_duration_s":0.001533,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000535,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000998,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01019,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014789,"completed_at":"2026-09-25T15:58:05.149Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1347gitlab # [ 210.890956] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.150Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4838d2138328fd9cca734024","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4838d2138328fd9cca734024: start","job_status":"start","queue_duration_s":1.836488,"scheduling_latency_s":1.836505}1348gitlab # [ 210.904633] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.165Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4838d2138328fd9cca734024","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4838d2138328fd9cca734024: done: 0.014534 sec","job_status":"done","queue_duration_s":1.836488,"scheduling_latency_s":1.836505,"redis_calls":13,"redis_duration_s":0.001484,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000549,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000935,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010049,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014534,"completed_at":"2026-09-25T15:58:05.165Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1349gitlab # [ 210.945110] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.166Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"82ecc9bf39c69725fbd6413a","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-82ecc9bf39c69725fbd6413a: start","job_status":"start","queue_duration_s":1.852114,"scheduling_latency_s":1.852132}1350gitlab # [ 210.958089] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.181Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"82ecc9bf39c69725fbd6413a","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-82ecc9bf39c69725fbd6413a: done: 0.01471 sec","job_status":"done","queue_duration_s":1.852114,"scheduling_latency_s":1.852132,"redis_calls":13,"redis_duration_s":0.001618,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001085,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009905,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01471,"completed_at":"2026-09-25T15:58:05.181Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1351gitlab # [ 210.998969] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.182Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1709cb2a3f9f86caf07039a2","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1709cb2a3f9f86caf07039a2: start","job_status":"start","queue_duration_s":1.867998,"scheduling_latency_s":1.868016}1352gitlab # [ 211.011611] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.197Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1709cb2a3f9f86caf07039a2","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1709cb2a3f9f86caf07039a2: done: 0.015086 sec","job_status":"done","queue_duration_s":1.867998,"scheduling_latency_s":1.868016,"redis_calls":13,"redis_duration_s":0.0015899999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000569,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001021,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010193,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015086,"completed_at":"2026-09-25T15:58:05.197Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1353gitlab # [ 211.051907] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.198Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5350c617032b5f455bc8bdf3","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5350c617032b5f455bc8bdf3: start","job_status":"start","queue_duration_s":1.884167,"scheduling_latency_s":1.884183}1354gitlab # [ 211.065325] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.212Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5350c617032b5f455bc8bdf3","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5350c617032b5f455bc8bdf3: done: 0.014554 sec","job_status":"done","queue_duration_s":1.884167,"scheduling_latency_s":1.884183,"redis_calls":13,"redis_duration_s":0.001645,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000534,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001111,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009856,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014554,"completed_at":"2026-09-25T15:58:05.212Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1355gitlab # [ 211.107742] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.214Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6cde14d5829faefa2573eee7","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6cde14d5829faefa2573eee7: start","job_status":"start","queue_duration_s":1.899805,"scheduling_latency_s":1.899823}1356gitlab # [ 211.120636] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.228Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6cde14d5829faefa2573eee7","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6cde14d5829faefa2573eee7: done: 0.014514 sec","job_status":"done","queue_duration_s":1.899805,"scheduling_latency_s":1.899823,"redis_calls":13,"redis_duration_s":0.001457,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000934,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00995,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014514,"completed_at":"2026-09-25T15:58:05.228Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1357gitlab # [ 211.160602] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.229Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"43280c21e8c732c2c4a1f173","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-43280c21e8c732c2c4a1f173: start","job_status":"start","queue_duration_s":1.915615,"scheduling_latency_s":1.915632}1358gitlab # [ 211.173415] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.244Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"43280c21e8c732c2c4a1f173","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-43280c21e8c732c2c4a1f173: done: 0.014666 sec","job_status":"done","queue_duration_s":1.915615,"scheduling_latency_s":1.915632,"redis_calls":13,"redis_duration_s":0.001488,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000957,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01013,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014666,"completed_at":"2026-09-25T15:58:05.244Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1359gitlab # [ 211.214329] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.245Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"925ea30da5e8240c3c104507","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-925ea30da5e8240c3c104507: start","job_status":"start","queue_duration_s":1.931538,"scheduling_latency_s":1.931555}1360gitlab # [ 211.227047] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.260Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"925ea30da5e8240c3c104507","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-925ea30da5e8240c3c104507: done: 0.014524 sec","job_status":"done","queue_duration_s":1.931538,"scheduling_latency_s":1.931555,"redis_calls":13,"redis_duration_s":0.001443,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000919,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010198,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014524,"completed_at":"2026-09-25T15:58:05.260Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1361gitlab # [ 211.266859] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.261Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"befebfbb934ef6ade8996432","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-befebfbb934ef6ade8996432: start","job_status":"start","queue_duration_s":1.947347,"scheduling_latency_s":1.947364}1362gitlab # [ 211.279667] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.276Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"befebfbb934ef6ade8996432","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-befebfbb934ef6ade8996432: done: 0.01451 sec","job_status":"done","queue_duration_s":1.947347,"scheduling_latency_s":1.947364,"redis_calls":13,"redis_duration_s":0.0014780000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000954,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010116,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01451,"completed_at":"2026-09-25T15:58:05.276Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1363gitlab # [ 211.319722] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.277Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e0994f04420d18649b5908df","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e0994f04420d18649b5908df: start","job_status":"start","queue_duration_s":1.963421,"scheduling_latency_s":1.963439}1364gitlab # [ 211.332588] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.292Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e0994f04420d18649b5908df","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e0994f04420d18649b5908df: done: 0.015083 sec","job_status":"done","queue_duration_s":1.963421,"scheduling_latency_s":1.963439,"redis_calls":13,"redis_duration_s":0.001507,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000981,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010485,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015083,"completed_at":"2026-09-25T15:58:05.292Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1365gitlab # [ 211.372459] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.294Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"097c8a275cce56a67d2e715b","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-097c8a275cce56a67d2e715b: start","job_status":"start","queue_duration_s":1.979907,"scheduling_latency_s":1.979925}1366gitlab # [ 211.385074] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.308Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"097c8a275cce56a67d2e715b","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-097c8a275cce56a67d2e715b: done: 0.014667 sec","job_status":"done","queue_duration_s":1.979907,"scheduling_latency_s":1.979925,"redis_calls":13,"redis_duration_s":0.0015040000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000544,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00096,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01016,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014667,"completed_at":"2026-09-25T15:58:05.308Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1367gitlab # [ 211.426807] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.309Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"56be0c57cebee5a60d2d3182","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-56be0c57cebee5a60d2d3182: start","job_status":"start","queue_duration_s":1.995733,"scheduling_latency_s":1.99575}1368gitlab # [ 211.439479] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.324Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"56be0c57cebee5a60d2d3182","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-56be0c57cebee5a60d2d3182: done: 0.014812 sec","job_status":"done","queue_duration_s":1.995733,"scheduling_latency_s":1.99575,"redis_calls":13,"redis_duration_s":0.0016740000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000538,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001136,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010178,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014812,"completed_at":"2026-09-25T15:58:05.324Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1369gitlab # [ 211.479840] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.325Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1e5e943c0593abfd8833b497","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1e5e943c0593abfd8833b497: start","job_status":"start","queue_duration_s":2.01167,"scheduling_latency_s":2.011687}1370gitlab # [ 211.492657] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.340Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1e5e943c0593abfd8833b497","created_at":"2026-09-25T15:58:03.039Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-f3867d8e60114e1b","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e5b7113ff9bc6c0632bb37f6913a5883","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.314Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1e5e943c0593abfd8833b497: done: 0.014776 sec","job_status":"done","queue_duration_s":2.01167,"scheduling_latency_s":2.011687,"redis_calls":13,"redis_duration_s":0.00168,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000583,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001097,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010037,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014776,"completed_at":"2026-09-25T15:58:05.340Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1371gitlab # [ 211.532803] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.341Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"5dd68d452eb975ac8d329a37","created_at":"2026-09-25T15:58:03.344Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-ac9721c9c95541f2","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8b8231c3d3830927977ed3eafe8e2a82","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.358Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-5dd68d452eb975ac8d329a37: start","job_status":"start","queue_duration_s":1.983073,"scheduling_latency_s":1.98309}1372gitlab # [ 211.545287] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.358Z","message":"No existing merge request to be cleaned up."}1373gitlab # [ 211.546946] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.371Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"5dd68d452eb975ac8d329a37","created_at":"2026-09-25T15:58:03.344Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-ac9721c9c95541f2","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8b8231c3d3830927977ed3eafe8e2a82","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.358Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-5dd68d452eb975ac8d329a37: done: 0.030099 sec","job_status":"done","queue_duration_s":1.983073,"scheduling_latency_s":1.98309,"redis_calls":13,"redis_duration_s":0.001959,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000307,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000677,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000975,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020249,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00027854,"duration_s":0.030099,"completed_at":"2026-09-25T15:58:05.371Z","load_balancing_strategy":"primary","db_duration_s":0.003778,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1374gitlab # [ 211.587780] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.373Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"3549d0ed093ef16c2069923d","created_at":"2026-09-25T15:58:03.344Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-ac9721c9c95541f2","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8b8231c3d3830927977ed3eafe8e2a82","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.358Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-3549d0ed093ef16c2069923d: start","job_status":"start","queue_duration_s":2.014461,"scheduling_latency_s":2.014478}1375gitlab # [ 211.600476] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.383Z","message":"No existing merge request to be cleaned up."}1376gitlab # [ 211.602175] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.391Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"3549d0ed093ef16c2069923d","created_at":"2026-09-25T15:58:03.344Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-ac9721c9c95541f2","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8b8231c3d3830927977ed3eafe8e2a82","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.358Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-3549d0ed093ef16c2069923d: done: 0.018253 sec","job_status":"done","queue_duration_s":2.014461,"scheduling_latency_s":2.014478,"redis_calls":10,"redis_duration_s":0.001529,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000765,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000764,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012983,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018253,"completed_at":"2026-09-25T15:58:05.391Z","load_balancing_strategy":"primary","db_duration_s":0.001493,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1377gitlab # [ 211.642272] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.392Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"7c485e84fe0322652bb29f67","created_at":"2026-09-25T15:58:03.344Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-ac9721c9c95541f2","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8b8231c3d3830927977ed3eafe8e2a82","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.358Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-7c485e84fe0322652bb29f67: start","job_status":"start","queue_duration_s":2.033989,"scheduling_latency_s":2.034006}1378gitlab # [ 211.654821] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.402Z","message":"No existing merge request to be cleaned up."}1379gitlab # [ 211.656548] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.410Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"7c485e84fe0322652bb29f67","created_at":"2026-09-25T15:58:03.344Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-ac9721c9c95541f2","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8b8231c3d3830927977ed3eafe8e2a82","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.358Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-7c485e84fe0322652bb29f67: done: 0.017871 sec","job_status":"done","queue_duration_s":2.033989,"scheduling_latency_s":2.034006,"redis_calls":10,"redis_duration_s":0.001349,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00057,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000779,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013013,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017871,"completed_at":"2026-09-25T15:58:05.410Z","load_balancing_strategy":"primary","db_duration_s":0.001241,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1380gitlab # [ 211.695712] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.411Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e43ab6769b1965da8bd301c2","created_at":"2026-09-25T15:58:03.344Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-ac9721c9c95541f2","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8b8231c3d3830927977ed3eafe8e2a82","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.358Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e43ab6769b1965da8bd301c2: start","job_status":"start","queue_duration_s":2.053203,"scheduling_latency_s":2.05322}1381gitlab # [ 211.708307] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.422Z","message":"No existing merge request to be cleaned up."}1382gitlab # [ 211.710455] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:58:05.429Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e43ab6769b1965da8bd301c2","created_at":"2026-09-25T15:58:03.344Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-ac9721c9c95541f2","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8b8231c3d3830927977ed3eafe8e2a82","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:58:03.358Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e43ab6769b1965da8bd301c2: done: 0.017868 sec","job_status":"done","queue_duration_s":2.053203,"scheduling_latency_s":2.05322,"redis_calls":10,"redis_duration_s":0.001352,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000554,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000798,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012837,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017868,"completed_at":"2026-09-25T15:58:05.429Z","load_balancing_strategy":"primary","db_duration_s":0.001214,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1383gitlab # [ 243.572933] setup-gitlab-start[2814]: W, [2026-09-25T15:58:39.463002 #2814] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1384gitlab # [ 265.324981] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:01.214Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"8d0aab62efcabc4dc3a85660","created_at":"2026-09-25T15:59:01.200Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0c253537d9222fb397e892ec4c9971f9","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.212Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-8d0aab62efcabc4dc3a85660: start","job_status":"start","queue_duration_s":0.002265,"scheduling_latency_s":0.002305}1385gitlab # [ 265.404780] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:01.294Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"8d0aab62efcabc4dc3a85660","created_at":"2026-09-25T15:59:01.200Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0c253537d9222fb397e892ec4c9971f9","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.212Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-8d0aab62efcabc4dc3a85660: done: 0.080416 sec","job_status":"done","queue_duration_s":0.002265,"scheduling_latency_s":0.002305,"redis_calls":16,"redis_duration_s":0.023818,"redis_read_bytes":1033,"redis_write_bytes":1343,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.01319,"redis_feature_flag_read_bytes":1024,"redis_feature_flag_write_bytes":513,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.008502,"redis_queues_metadata_read_bytes":8,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002126,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027188,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.080416,"completed_at":"2026-09-25T15:59:01.294Z","load_balancing_strategy":"primary","db_duration_s":0.009685,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1386gitlab # [ 265.471255] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:01.310Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"7e983c5a92303f770f93bc51","created_at":"2026-09-25T15:59:01.225Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.232Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-7e983c5a92303f770f93bc51: start","job_status":"start","queue_duration_s":0.077387,"scheduling_latency_s":0.077419}1387gitlab # [ 265.905077] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:01.795Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"7e983c5a92303f770f93bc51","created_at":"2026-09-25T15:59:01.225Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.232Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-7e983c5a92303f770f93bc51: done: 0.485136 sec","job_status":"done","queue_duration_s":0.077387,"scheduling_latency_s":0.077419,"redis_calls":60,"redis_duration_s":0.183324,"redis_read_bytes":108,"redis_write_bytes":41059,"redis_queues_calls":2,"redis_queues_duration_s":0.002235,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.177314,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.003775,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.142674,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.485136,"completed_at":"2026-09-25T15:59:01.795Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1388gitlab # [ 265.962673] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:01.797Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"d21aed8e9c3fde546c0d6a57","created_at":"2026-09-25T15:59:01.247Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"53f6687ecfff1d73ff278f0aa3a59b3e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.258Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-d21aed8e9c3fde546c0d6a57: start","job_status":"start","queue_duration_s":0.539214,"scheduling_latency_s":0.539243}1389gitlab # [ 265.977395] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:01.832Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"d21aed8e9c3fde546c0d6a57","created_at":"2026-09-25T15:59:01.247Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"53f6687ecfff1d73ff278f0aa3a59b3e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.258Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-d21aed8e9c3fde546c0d6a57: done: 0.035211 sec","job_status":"done","queue_duration_s":0.539214,"scheduling_latency_s":0.539243,"redis_calls":14,"redis_duration_s":0.003083,"redis_read_bytes":16,"redis_write_bytes":4026,"redis_queues_calls":2,"redis_queues_duration_s":0.000369,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002015,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000699,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02628,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.035211,"completed_at":"2026-09-25T15:59:01.832Z","load_balancing_strategy":"primary","db_duration_s":0.001962,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1390gitlab # [ 266.019071] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:01.834Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"04d28358f51a9db892c1fd1d","created_at":"2026-09-25T15:59:01.264Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9aef41c1b387659bed48ea7aaeb51b06","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.272Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-04d28358f51a9db892c1fd1d: start","job_status":"start","queue_duration_s":0.561344,"scheduling_latency_s":0.561367}1391gitlab # [ 266.034285] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:01.844Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1392gitlab # [ 266.037740] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:01.850Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"04d28358f51a9db892c1fd1d","created_at":"2026-09-25T15:59:01.264Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9aef41c1b387659bed48ea7aaeb51b06","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.272Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-04d28358f51a9db892c1fd1d: done: 0.016342 sec","job_status":"done","queue_duration_s":0.561344,"scheduling_latency_s":0.561367,"redis_calls":7,"redis_duration_s":0.0015,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001078,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000422,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010434,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016342,"completed_at":"2026-09-25T15:59:01.850Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1393gitlab # [ 266.079144] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:01.852Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"4a12439e579f9d32c65467f3","created_at":"2026-09-25T15:59:01.283Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9a3cf610ae89bd5fd175d0308a99e272","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.290Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-4a12439e579f9d32c65467f3: start","job_status":"start","queue_duration_s":0.56162,"scheduling_latency_s":0.561641}1394gitlab # [ 266.093908] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:01.860Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1395gitlab # [ 266.097627] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:01.865Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"4a12439e579f9d32c65467f3","created_at":"2026-09-25T15:59:01.283Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9a3cf610ae89bd5fd175d0308a99e272","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.290Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-4a12439e579f9d32c65467f3: done: 0.013858 sec","job_status":"done","queue_duration_s":0.56162,"scheduling_latency_s":0.561641,"redis_calls":7,"redis_duration_s":0.001268,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000902,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000366,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008737,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013858,"completed_at":"2026-09-25T15:59:01.865Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1396gitlab # [ 266.139889] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:01.867Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"33d7ea5707b69a8dbe397334","created_at":"2026-09-25T15:59:01.302Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2877f896b43bf21e04fa33a584d9a270","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.308Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-33d7ea5707b69a8dbe397334: start","job_status":"start","queue_duration_s":0.559876,"scheduling_latency_s":0.559905}1397gitlab # [ 266.154853] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:01.878Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1398gitlab # [ 266.158901] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:01.884Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"33d7ea5707b69a8dbe397334","created_at":"2026-09-25T15:59:01.302Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2877f896b43bf21e04fa33a584d9a270","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.308Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-33d7ea5707b69a8dbe397334: done: 0.016955 sec","job_status":"done","queue_duration_s":0.559876,"scheduling_latency_s":0.559905,"redis_calls":7,"redis_duration_s":0.001465,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001076,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000389,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01084,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016955,"completed_at":"2026-09-25T15:59:01.884Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1399gitlab # [ 266.200582] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:01.886Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"758873cf67162227254642fa","created_at":"2026-09-25T15:59:01.322Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"33b91e1b0fb64200163de9c76e72d63c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.326Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-758873cf67162227254642fa: start","job_status":"start","queue_duration_s":0.559847,"scheduling_latency_s":0.55987}1400gitlab # [ 266.215279] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:01.896Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1401gitlab # [ 266.218673] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:01.902Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"758873cf67162227254642fa","created_at":"2026-09-25T15:59:01.322Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"33b91e1b0fb64200163de9c76e72d63c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.326Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-758873cf67162227254642fa: done: 0.016342 sec","job_status":"done","queue_duration_s":0.559847,"scheduling_latency_s":0.55987,"redis_calls":7,"redis_duration_s":0.001431,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001018,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000413,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010551,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016342,"completed_at":"2026-09-25T15:59:01.902Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1402gitlab # [ 266.260126] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:01.904Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"4251aec2d09886dceef0c99e","created_at":"2026-09-25T15:59:01.343Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eb575a7b4a4bd14189202ff724277bdc","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.351Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-4251aec2d09886dceef0c99e: start","job_status":"start","queue_duration_s":0.553222,"scheduling_latency_s":0.553246}1403gitlab # [ 266.275105] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:01.957Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"4251aec2d09886dceef0c99e","created_at":"2026-09-25T15:59:01.343Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eb575a7b4a4bd14189202ff724277bdc","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.351Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-4251aec2d09886dceef0c99e: done: 0.052585 sec","job_status":"done","queue_duration_s":0.553222,"scheduling_latency_s":0.553246,"redis_calls":9,"redis_duration_s":0.002313,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001125,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001188,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024923,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00091042,"exclusive_lock_hold_duration_s":0.01996966,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.052585,"completed_at":"2026-09-25T15:59:01.957Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.022255,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1404gitlab # [ 266.320058] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:01.959Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"de5eb14d3a0402ae3d3f741d","created_at":"2026-09-25T15:59:01.380Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"931295b21c7e824a1fdd7d6acf014d03","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.384Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-de5eb14d3a0402ae3d3f741d: start","job_status":"start","queue_duration_s":0.574938,"scheduling_latency_s":0.574965}1405gitlab # [ 266.335765] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:01.983Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"de5eb14d3a0402ae3d3f741d","created_at":"2026-09-25T15:59:01.380Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"931295b21c7e824a1fdd7d6acf014d03","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.384Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-de5eb14d3a0402ae3d3f741d: done: 0.024262 sec","job_status":"done","queue_duration_s":0.574938,"scheduling_latency_s":0.574965,"redis_calls":7,"redis_duration_s":0.001544,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00114,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000404,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014705,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024262,"completed_at":"2026-09-25T15:59:01.983Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003725,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1406gitlab # [ 266.377326] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:01.985Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"87fa015162f1f90ec25cdd8b","created_at":"2026-09-25T15:59:01.402Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0f16201839b8036cf5bec957d3ba2708","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.411Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-87fa015162f1f90ec25cdd8b: start","job_status":"start","queue_duration_s":0.573673,"scheduling_latency_s":0.573696}1407gitlab # [ 266.392130] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.010Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"87fa015162f1f90ec25cdd8b","created_at":"2026-09-25T15:59:01.402Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0f16201839b8036cf5bec957d3ba2708","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.411Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-87fa015162f1f90ec25cdd8b: done: 0.025807 sec","job_status":"done","queue_duration_s":0.573673,"scheduling_latency_s":0.573696,"redis_calls":10,"redis_duration_s":0.002345,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001258,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001087,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01724,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00604902,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.025807,"completed_at":"2026-09-25T15:59:02.010Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002688,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1408gitlab # [ 266.435444] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.012Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Gitlab::JiraImport::StuckJiraImportJobsWorker","jid":"4d5b129c5cb8323c19981c17","created_at":"2026-09-25T15:59:01.431Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e3e15cc78031d5f8f859f1354bd295c1","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b9cf6c9b6c2f8fad80e08693d59af787bb65ad6ffe24ec5d0e1a669517566127","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.437Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Gitlab::JiraImport::StuckJiraImportJobsWorker JID-4d5b129c5cb8323c19981c17: start","job_status":"start","queue_duration_s":0.575138,"scheduling_latency_s":0.575159}1409gitlab # [ 266.449415] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.032Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Gitlab::JiraImport::StuckJiraImportJobsWorker","jid":"4d5b129c5cb8323c19981c17","created_at":"2026-09-25T15:59:01.431Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e3e15cc78031d5f8f859f1354bd295c1","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b9cf6c9b6c2f8fad80e08693d59af787bb65ad6ffe24ec5d0e1a669517566127","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.437Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Gitlab::JiraImport::StuckJiraImportJobsWorker JID-4d5b129c5cb8323c19981c17: done: 0.019965 sec","job_status":"done","queue_duration_s":0.575138,"scheduling_latency_s":0.575159,"redis_calls":7,"redis_duration_s":0.001458,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001064,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000394,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013134,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019965,"completed_at":"2026-09-25T15:59:02.032Z","load_balancing_strategy":"primary","db_duration_s":0.001591,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1410gitlab # [ 266.489858] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.033Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"f38138da5a3d0b01a8a49515","created_at":"2026-09-25T15:59:01.451Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8e9e83fb277ce24e0ee2a69ff8dcbc4a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.458Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-f38138da5a3d0b01a8a49515: start","job_status":"start","queue_duration_s":0.575646,"scheduling_latency_s":0.575667}1411gitlab # [ 266.505309] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.049Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"f38138da5a3d0b01a8a49515","created_at":"2026-09-25T15:59:01.451Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8e9e83fb277ce24e0ee2a69ff8dcbc4a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.458Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-f38138da5a3d0b01a8a49515: done: 0.015358 sec","job_status":"done","queue_duration_s":0.575646,"scheduling_latency_s":0.575667,"redis_calls":8,"redis_duration_s":0.0014449999999999999,"redis_read_bytes":133,"redis_write_bytes":901,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000171,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":70,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000913,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000361,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010129,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015358,"completed_at":"2026-09-25T15:59:02.049Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1412gitlab # [ 266.549890] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.050Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"e04da393ab24b2e59f1b45c8","created_at":"2026-09-25T15:59:01.508Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6398b32732ffcea75d5444f0c42d6375","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-25T15:59:01.513Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-e04da393ab24b2e59f1b45c8: start","job_status":"start","queue_duration_s":0.536672,"scheduling_latency_s":0.53669}1413gitlab # [ 266.563539] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.071Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"e04da393ab24b2e59f1b45c8","created_at":"2026-09-25T15:59:01.508Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6398b32732ffcea75d5444f0c42d6375","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-25T15:59:01.513Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-e04da393ab24b2e59f1b45c8: done: 0.020549 sec","job_status":"done","queue_duration_s":0.536672,"scheduling_latency_s":0.53669,"redis_calls":7,"redis_duration_s":0.001343,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000979,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000364,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014168,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020549,"completed_at":"2026-09-25T15:59:02.071Z","load_balancing_strategy":"primary","db_duration_s":0.001668,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1414gitlab # [ 266.605595] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.072Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"dd6fa0df59950fa60943c860","created_at":"2026-09-25T15:59:01.529Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"202eca2b91bb0c6d658bd7c441a33eff","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-25T15:59:01.535Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-dd6fa0df59950fa60943c860: start","job_status":"start","queue_duration_s":0.536869,"scheduling_latency_s":0.536884}1415gitlab # [ 266.620066] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.079Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1416gitlab # [ 266.623658] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.084Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"dd6fa0df59950fa60943c860","created_at":"2026-09-25T15:59:01.529Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"202eca2b91bb0c6d658bd7c441a33eff","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-25T15:59:01.535Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-dd6fa0df59950fa60943c860: done: 0.012121 sec","job_status":"done","queue_duration_s":0.536869,"scheduling_latency_s":0.536884,"redis_calls":7,"redis_duration_s":0.001163,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000835,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000328,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007614,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012121,"completed_at":"2026-09-25T15:59:02.084Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1417gitlab # [ 266.665682] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.085Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"c27a54c86255c6b820757aeb","created_at":"2026-09-25T15:59:01.544Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ed97a89dd73d1535b4113ab1f2de7983","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.552Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-c27a54c86255c6b820757aeb: start","job_status":"start","queue_duration_s":0.532796,"scheduling_latency_s":0.532813}1418gitlab # [ 266.680230] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.100Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"c27a54c86255c6b820757aeb","created_at":"2026-09-25T15:59:01.544Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ed97a89dd73d1535b4113ab1f2de7983","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.552Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-c27a54c86255c6b820757aeb: done: 0.01432 sec","job_status":"done","queue_duration_s":0.532796,"scheduling_latency_s":0.532813,"redis_calls":7,"redis_duration_s":0.001304,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000963,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000341,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008881,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01432,"completed_at":"2026-09-25T15:59:02.100Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001056,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1419gitlab # [ 266.722066] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.101Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"d033faab4cbec0160bac9d42","created_at":"2026-09-25T15:59:01.568Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5dfb5e4c027108f5b4e25a4cfe403d40","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.574Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-d033faab4cbec0160bac9d42: start","job_status":"start","queue_duration_s":0.52712,"scheduling_latency_s":0.527136}1420gitlab # [ 266.737386] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.116Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"d033faab4cbec0160bac9d42","created_at":"2026-09-25T15:59:01.568Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5dfb5e4c027108f5b4e25a4cfe403d40","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.574Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-d033faab4cbec0160bac9d42: done: 0.015067 sec","job_status":"done","queue_duration_s":0.52712,"scheduling_latency_s":0.527136,"redis_calls":9,"redis_duration_s":0.001698,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000876,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000822,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009399,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0004731,"exclusive_lock_hold_duration_s":0.00265866,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.015067,"completed_at":"2026-09-25T15:59:02.116Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001142,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1421gitlab # [ 266.780601] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.117Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"5d49485b338450b3880a71e0","created_at":"2026-09-25T15:59:01.585Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f5a32cf6ae1e237f6f583b5c68eddecc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.591Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-5d49485b338450b3880a71e0: start","job_status":"start","queue_duration_s":0.52585,"scheduling_latency_s":0.525867}1422gitlab # [ 266.794956] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.350Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"5d49485b338450b3880a71e0","created_at":"2026-09-25T15:59:01.585Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f5a32cf6ae1e237f6f583b5c68eddecc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.591Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-5d49485b338450b3880a71e0: done: 0.233275 sec","job_status":"done","queue_duration_s":0.52585,"scheduling_latency_s":0.525867,"redis_calls":9,"redis_duration_s":0.001727,"redis_read_bytes":7,"redis_write_bytes":987,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000911,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000816,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":129,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":129,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.032,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.032,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.187934,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00046812,"exclusive_lock_hold_duration_s":0.21941074,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.233275,"completed_at":"2026-09-25T15:59:02.350Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.059358,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1423gitlab # [ 266.838722] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.352Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"bab11de961b759cca4a8fa4f","created_at":"2026-09-25T15:59:01.608Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2472ab2c5cd9e7f9981a3d11b3e8faba","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.614Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-bab11de961b759cca4a8fa4f: start","job_status":"start","queue_duration_s":0.738179,"scheduling_latency_s":0.7382}1424gitlab # [ 266.853552] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.658Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"bab11de961b759cca4a8fa4f","created_at":"2026-09-25T15:59:01.608Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2472ab2c5cd9e7f9981a3d11b3e8faba","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.614Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-bab11de961b759cca4a8fa4f: done: 0.306577 sec","job_status":"done","queue_duration_s":0.738179,"scheduling_latency_s":0.7382,"redis_calls":562,"redis_duration_s":0.062178,"redis_read_bytes":560,"redis_write_bytes":48760,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000886,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.061292,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.277655,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.306577,"completed_at":"2026-09-25T15:59:02.658Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1425gitlab # [ 266.895029] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.660Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"16680813c6a97c70e14ba5b1","created_at":"2026-09-25T15:59:01.627Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6b8a6d713374a061d13c6d7f7d16b186","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.632Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-16680813c6a97c70e14ba5b1: start","job_status":"start","queue_duration_s":1.027503,"scheduling_latency_s":1.027528}1426gitlab # [ 266.909954] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.673Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"16680813c6a97c70e14ba5b1","created_at":"2026-09-25T15:59:01.627Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6b8a6d713374a061d13c6d7f7d16b186","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.632Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-16680813c6a97c70e14ba5b1: done: 0.013553 sec","job_status":"done","queue_duration_s":1.027503,"scheduling_latency_s":1.027528,"redis_calls":7,"redis_duration_s":0.001001,"redis_read_bytes":5,"redis_write_bytes":776,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000712,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000289,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008577,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013553,"completed_at":"2026-09-25T15:59:02.673Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1427gitlab # [ 266.953098] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.675Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"a80df431a674c5f752c01c76","created_at":"2026-09-25T15:59:01.669Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d6209897289cde75973da2ad2dbd3d3d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.675Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-a80df431a674c5f752c01c76: start","job_status":"start","queue_duration_s":1.000052,"scheduling_latency_s":1.00007}1428gitlab # [ 266.967009] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.691Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"a80df431a674c5f752c01c76","created_at":"2026-09-25T15:59:01.669Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d6209897289cde75973da2ad2dbd3d3d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.675Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-a80df431a674c5f752c01c76: done: 0.016385 sec","job_status":"done","queue_duration_s":1.000052,"scheduling_latency_s":1.00007,"redis_calls":9,"redis_duration_s":0.001302,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00066,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000642,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010942,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00051052,"exclusive_lock_hold_duration_s":0.00363554,"duration_s":0.016385,"completed_at":"2026-09-25T15:59:02.691Z","load_balancing_strategy":"primary","db_duration_s":0.001166,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1429gitlab # [ 267.007884] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.692Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"4b974c0944fab53330ad6fc7","created_at":"2026-09-25T15:59:01.713Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0304031c4d6d1c9cb609cf674ddd7078","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-25T15:59:01.727Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-4b974c0944fab53330ad6fc7: start","job_status":"start","queue_duration_s":0.965116,"scheduling_latency_s":0.965133}1430gitlab # [ 267.022529] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.699Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1431gitlab # [ 267.026723] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.704Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"4b974c0944fab53330ad6fc7","created_at":"2026-09-25T15:59:01.713Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0304031c4d6d1c9cb609cf674ddd7078","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-25T15:59:01.727Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-4b974c0944fab53330ad6fc7: done: 0.011847 sec","job_status":"done","queue_duration_s":0.965116,"scheduling_latency_s":0.965133,"redis_calls":7,"redis_duration_s":0.0009609999999999999,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000685,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000276,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007695,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011847,"completed_at":"2026-09-25T15:59:02.704Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1432gitlab # [ 267.068864] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.705Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7b5d4bca23f7e11798aeb388","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7b5d4bca23f7e11798aeb388: start","job_status":"start","queue_duration_s":0.96284,"scheduling_latency_s":0.962858}1433gitlab # [ 267.081709] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.719Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7b5d4bca23f7e11798aeb388","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7b5d4bca23f7e11798aeb388: done: 0.014105 sec","job_status":"done","queue_duration_s":0.96284,"scheduling_latency_s":0.962858,"redis_calls":13,"redis_duration_s":0.00146,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000515,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000945,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009621,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014105,"completed_at":"2026-09-25T15:59:02.719Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1434gitlab # [ 267.121487] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.720Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e7b7f064a95d133f52d535ef","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e7b7f064a95d133f52d535ef: start","job_status":"start","queue_duration_s":0.977989,"scheduling_latency_s":0.978006}1435gitlab # [ 267.135028] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.736Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e7b7f064a95d133f52d535ef","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e7b7f064a95d133f52d535ef: done: 0.015923 sec","job_status":"done","queue_duration_s":0.977989,"scheduling_latency_s":0.978006,"redis_calls":13,"redis_duration_s":0.0017980000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000684,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001114,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011043,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015923,"completed_at":"2026-09-25T15:59:02.736Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1436gitlab # [ 267.176068] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.738Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cb7e09be5c472ab1ac0b6f95","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cb7e09be5c472ab1ac0b6f95: start","job_status":"start","queue_duration_s":0.995656,"scheduling_latency_s":0.995681}1437gitlab # [ 267.188707] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.757Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cb7e09be5c472ab1ac0b6f95","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cb7e09be5c472ab1ac0b6f95: done: 0.018925 sec","job_status":"done","queue_duration_s":0.995656,"scheduling_latency_s":0.995681,"redis_calls":13,"redis_duration_s":0.002156,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000787,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001369,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013419,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018925,"completed_at":"2026-09-25T15:59:02.757Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1438gitlab # [ 267.228983] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.758Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5ca09e17b4d35a94f560e6bb","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5ca09e17b4d35a94f560e6bb: start","job_status":"start","queue_duration_s":1.01565,"scheduling_latency_s":1.015668}1439gitlab # [ 267.241778] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.796Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5ca09e17b4d35a94f560e6bb","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5ca09e17b4d35a94f560e6bb: done: 0.038651 sec","job_status":"done","queue_duration_s":1.01565,"scheduling_latency_s":1.015668,"redis_calls":13,"redis_duration_s":0.0014810000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000561,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00092,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010614,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.038651,"completed_at":"2026-09-25T15:59:02.796Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1440gitlab # [ 267.281673] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.798Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"311b33aeda28d614c7f9acfa","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-311b33aeda28d614c7f9acfa: start","job_status":"start","queue_duration_s":1.055364,"scheduling_latency_s":1.05538}1441gitlab # [ 267.294823] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.812Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"311b33aeda28d614c7f9acfa","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-311b33aeda28d614c7f9acfa: done: 0.014739 sec","job_status":"done","queue_duration_s":1.055364,"scheduling_latency_s":1.05538,"redis_calls":13,"redis_duration_s":0.0014880000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000938,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010462,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014739,"completed_at":"2026-09-25T15:59:02.812Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1442gitlab # [ 267.337392] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.813Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a7fd49f737edc411687c2b93","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a7fd49f737edc411687c2b93: start","job_status":"start","queue_duration_s":1.07116,"scheduling_latency_s":1.071177}1443gitlab # [ 267.350298] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.828Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a7fd49f737edc411687c2b93","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a7fd49f737edc411687c2b93: done: 0.014714 sec","job_status":"done","queue_duration_s":1.07116,"scheduling_latency_s":1.071177,"redis_calls":13,"redis_duration_s":0.002148,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001216,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000932,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01038,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014714,"completed_at":"2026-09-25T15:59:02.828Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1444gitlab # [ 267.390280] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.829Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8c45cbf3867880c59c67e2cb","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8c45cbf3867880c59c67e2cb: start","job_status":"start","queue_duration_s":1.086992,"scheduling_latency_s":1.087008}1445gitlab # [ 267.402976] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.844Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8c45cbf3867880c59c67e2cb","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8c45cbf3867880c59c67e2cb: done: 0.014595 sec","job_status":"done","queue_duration_s":1.086992,"scheduling_latency_s":1.087008,"redis_calls":13,"redis_duration_s":0.001431,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000503,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000928,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009213,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014595,"completed_at":"2026-09-25T15:59:02.844Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1446gitlab # [ 267.443090] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.846Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"201365d09b52eb13c7851811","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-201365d09b52eb13c7851811: start","job_status":"start","queue_duration_s":1.103476,"scheduling_latency_s":1.103496}1447gitlab # [ 267.456230] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.860Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"201365d09b52eb13c7851811","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-201365d09b52eb13c7851811: done: 0.014547 sec","job_status":"done","queue_duration_s":1.103476,"scheduling_latency_s":1.103496,"redis_calls":13,"redis_duration_s":0.001415,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000515,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0009,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009183,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014547,"completed_at":"2026-09-25T15:59:02.860Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1448gitlab # [ 267.496424] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.861Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7aaeefd1949769c94704e1f2","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7aaeefd1949769c94704e1f2: start","job_status":"start","queue_duration_s":1.119084,"scheduling_latency_s":1.119101}1449gitlab # [ 267.509044] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.876Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7aaeefd1949769c94704e1f2","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7aaeefd1949769c94704e1f2: done: 0.014655 sec","job_status":"done","queue_duration_s":1.119084,"scheduling_latency_s":1.119101,"redis_calls":13,"redis_duration_s":0.001453,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00093,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010434,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014655,"completed_at":"2026-09-25T15:59:02.876Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1450gitlab # [ 267.550249] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.877Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68eeb10b24ca3ea34abaf5a4","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68eeb10b24ca3ea34abaf5a4: start","job_status":"start","queue_duration_s":1.134785,"scheduling_latency_s":1.134801}1451gitlab # [ 267.563434] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.891Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68eeb10b24ca3ea34abaf5a4","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68eeb10b24ca3ea34abaf5a4: done: 0.014421 sec","job_status":"done","queue_duration_s":1.134785,"scheduling_latency_s":1.134801,"redis_calls":13,"redis_duration_s":0.001461,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000548,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000913,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010243,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014421,"completed_at":"2026-09-25T15:59:02.891Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1452gitlab # [ 267.604292] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.892Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9366236731272b9375baef75","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9366236731272b9375baef75: start","job_status":"start","queue_duration_s":1.150261,"scheduling_latency_s":1.150277}1453gitlab # [ 267.617441] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.907Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9366236731272b9375baef75","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9366236731272b9375baef75: done: 0.014222 sec","job_status":"done","queue_duration_s":1.150261,"scheduling_latency_s":1.150277,"redis_calls":13,"redis_duration_s":0.0014069999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000899,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009159,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014222,"completed_at":"2026-09-25T15:59:02.907Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1454gitlab # [ 267.657900] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.909Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"28b6b0afc317b378d257bc46","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-28b6b0afc317b378d257bc46: start","job_status":"start","queue_duration_s":1.166411,"scheduling_latency_s":1.166428}1455gitlab # [ 267.670456] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.923Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"28b6b0afc317b378d257bc46","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-28b6b0afc317b378d257bc46: done: 0.014616 sec","job_status":"done","queue_duration_s":1.166411,"scheduling_latency_s":1.166428,"redis_calls":13,"redis_duration_s":0.001458,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000522,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000936,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010346,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014616,"completed_at":"2026-09-25T15:59:02.923Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1456gitlab # [ 267.710638] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.924Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2713ebba133c537e432acffd","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2713ebba133c537e432acffd: start","job_status":"start","queue_duration_s":1.182039,"scheduling_latency_s":1.182055}1457gitlab # [ 267.723540] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.939Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2713ebba133c537e432acffd","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2713ebba133c537e432acffd: done: 0.014645 sec","job_status":"done","queue_duration_s":1.182039,"scheduling_latency_s":1.182055,"redis_calls":13,"redis_duration_s":0.001475,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000517,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000958,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010323,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014645,"completed_at":"2026-09-25T15:59:02.939Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1458gitlab # [ 267.764606] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.940Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5d8ab91770e9ffdde7ea2c5b","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5d8ab91770e9ffdde7ea2c5b: start","job_status":"start","queue_duration_s":1.197733,"scheduling_latency_s":1.197749}1459gitlab # [ 267.777120] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.955Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5d8ab91770e9ffdde7ea2c5b","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5d8ab91770e9ffdde7ea2c5b: done: 0.015002 sec","job_status":"done","queue_duration_s":1.197733,"scheduling_latency_s":1.197749,"redis_calls":13,"redis_duration_s":0.001449,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000922,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010548,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015002,"completed_at":"2026-09-25T15:59:02.955Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1460gitlab # [ 267.816844] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.956Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"37ca34858c4bc8e3f70bbd82","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-37ca34858c4bc8e3f70bbd82: start","job_status":"start","queue_duration_s":1.213769,"scheduling_latency_s":1.213785}1461gitlab # [ 267.829677] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.971Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"37ca34858c4bc8e3f70bbd82","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-37ca34858c4bc8e3f70bbd82: done: 0.015362 sec","job_status":"done","queue_duration_s":1.213769,"scheduling_latency_s":1.213785,"redis_calls":13,"redis_duration_s":0.0014190000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000509,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00091,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009999,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015362,"completed_at":"2026-09-25T15:59:02.971Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1462gitlab # [ 267.871606] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.973Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4c27dd7b53b9c1c87d7fffae","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4c27dd7b53b9c1c87d7fffae: start","job_status":"start","queue_duration_s":1.231047,"scheduling_latency_s":1.231072}1463gitlab # [ 267.884501] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.994Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4c27dd7b53b9c1c87d7fffae","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4c27dd7b53b9c1c87d7fffae: done: 0.02104 sec","job_status":"done","queue_duration_s":1.231047,"scheduling_latency_s":1.231072,"redis_calls":13,"redis_duration_s":0.002395,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000788,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001607,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014991,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02104,"completed_at":"2026-09-25T15:59:02.994Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1464gitlab # [ 267.925645] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:02.996Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3b52da9ed908fb0678768857","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3b52da9ed908fb0678768857: start","job_status":"start","queue_duration_s":1.253536,"scheduling_latency_s":1.253557}1465gitlab # [ 267.938477] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.016Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3b52da9ed908fb0678768857","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3b52da9ed908fb0678768857: done: 0.020748 sec","job_status":"done","queue_duration_s":1.253536,"scheduling_latency_s":1.253557,"redis_calls":14,"redis_duration_s":0.002352,"redis_read_bytes":138,"redis_write_bytes":1216,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000265,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":59,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000737,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00135,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014881,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020748,"completed_at":"2026-09-25T15:59:03.016Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1466gitlab # [ 267.982532] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.018Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"24369116c0dbc6527ef56d6f","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-24369116c0dbc6527ef56d6f: start","job_status":"start","queue_duration_s":1.275943,"scheduling_latency_s":1.275968}1467gitlab # [ 267.995153] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.040Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"24369116c0dbc6527ef56d6f","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-24369116c0dbc6527ef56d6f: done: 0.021858 sec","job_status":"done","queue_duration_s":1.275943,"scheduling_latency_s":1.275968,"redis_calls":14,"redis_duration_s":0.002433,"redis_read_bytes":138,"redis_write_bytes":1218,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000188,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":61,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000825,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00142,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014617,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021858,"completed_at":"2026-09-25T15:59:03.040Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1468gitlab # [ 268.036923] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.041Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c6904fa6095d7c3be2943073","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c6904fa6095d7c3be2943073: start","job_status":"start","queue_duration_s":1.299141,"scheduling_latency_s":1.29916}1469gitlab # [ 268.049970] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.059Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c6904fa6095d7c3be2943073","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c6904fa6095d7c3be2943073: done: 0.018043 sec","job_status":"done","queue_duration_s":1.299141,"scheduling_latency_s":1.29916,"redis_calls":14,"redis_duration_s":0.002442,"redis_read_bytes":138,"redis_write_bytes":1221,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000187,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":64,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000735,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00152,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012546,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018043,"completed_at":"2026-09-25T15:59:03.059Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1470gitlab # [ 268.092617] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.061Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8ea3099dbaad6348a4457d4d","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8ea3099dbaad6348a4457d4d: start","job_status":"start","queue_duration_s":1.318409,"scheduling_latency_s":1.318427}1471gitlab # [ 268.105474] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.078Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8ea3099dbaad6348a4457d4d","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8ea3099dbaad6348a4457d4d: done: 0.017564 sec","job_status":"done","queue_duration_s":1.318409,"scheduling_latency_s":1.318427,"redis_calls":14,"redis_duration_s":0.00208,"redis_read_bytes":138,"redis_write_bytes":1269,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000169,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":112,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000657,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001254,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010516,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017564,"completed_at":"2026-09-25T15:59:03.078Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1472gitlab # [ 268.149122] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.079Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fb4b9372bb776af07b9d045f","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fb4b9372bb776af07b9d045f: start","job_status":"start","queue_duration_s":1.337198,"scheduling_latency_s":1.337216}1473gitlab # [ 268.162162] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.095Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fb4b9372bb776af07b9d045f","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fb4b9372bb776af07b9d045f: done: 0.016072 sec","job_status":"done","queue_duration_s":1.337198,"scheduling_latency_s":1.337216,"redis_calls":13,"redis_duration_s":0.002101,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000712,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001389,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011157,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016072,"completed_at":"2026-09-25T15:59:03.095Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1474gitlab # [ 268.203698] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.097Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"db8bf5b1844b1b4ddaa3d152","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-db8bf5b1844b1b4ddaa3d152: start","job_status":"start","queue_duration_s":1.354438,"scheduling_latency_s":1.354455}1475gitlab # [ 268.217240] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.111Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"db8bf5b1844b1b4ddaa3d152","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-db8bf5b1844b1b4ddaa3d152: done: 0.014536 sec","job_status":"done","queue_duration_s":1.354438,"scheduling_latency_s":1.354455,"redis_calls":13,"redis_duration_s":0.002634,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001462,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001172,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009938,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014536,"completed_at":"2026-09-25T15:59:03.111Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1476gitlab # [ 268.257142] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.112Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dd7d61c9519ee942d4f1f5b5","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dd7d61c9519ee942d4f1f5b5: start","job_status":"start","queue_duration_s":1.370097,"scheduling_latency_s":1.370113}1477gitlab # [ 268.269659] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.126Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dd7d61c9519ee942d4f1f5b5","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dd7d61c9519ee942d4f1f5b5: done: 0.014104 sec","job_status":"done","queue_duration_s":1.370097,"scheduling_latency_s":1.370113,"redis_calls":13,"redis_duration_s":0.001861,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000645,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001216,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008705,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014104,"completed_at":"2026-09-25T15:59:03.126Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1478gitlab # [ 268.309484] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.128Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"74f24eb48afff4bb33c4132f","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-74f24eb48afff4bb33c4132f: start","job_status":"start","queue_duration_s":1.386288,"scheduling_latency_s":1.386303}1479gitlab # [ 268.322700] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.143Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"74f24eb48afff4bb33c4132f","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-74f24eb48afff4bb33c4132f: done: 0.014367 sec","job_status":"done","queue_duration_s":1.386288,"scheduling_latency_s":1.386303,"redis_calls":13,"redis_duration_s":0.0018770000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000635,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001242,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009706,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014367,"completed_at":"2026-09-25T15:59:03.143Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1480gitlab # [ 268.364882] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.144Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"df176d4565347093ef532f8c","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-df176d4565347093ef532f8c: start","job_status":"start","queue_duration_s":1.40179,"scheduling_latency_s":1.401806}1481gitlab # [ 268.378239] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.159Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"df176d4565347093ef532f8c","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-df176d4565347093ef532f8c: done: 0.014618 sec","job_status":"done","queue_duration_s":1.40179,"scheduling_latency_s":1.401806,"redis_calls":13,"redis_duration_s":0.001935,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00068,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001255,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010049,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014618,"completed_at":"2026-09-25T15:59:03.159Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1482gitlab # [ 268.418846] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.160Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aaeddcda639be5b1fd0f56cf","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aaeddcda639be5b1fd0f56cf: start","job_status":"start","queue_duration_s":1.417569,"scheduling_latency_s":1.417585}1483gitlab # [ 268.432416] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.174Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aaeddcda639be5b1fd0f56cf","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aaeddcda639be5b1fd0f56cf: done: 0.01438 sec","job_status":"done","queue_duration_s":1.417569,"scheduling_latency_s":1.417585,"redis_calls":13,"redis_duration_s":0.0018499999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000627,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001223,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008702,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01438,"completed_at":"2026-09-25T15:59:03.174Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1484gitlab # [ 268.474051] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.176Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b77041688a3518870fe49f64","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b77041688a3518870fe49f64: start","job_status":"start","queue_duration_s":1.434175,"scheduling_latency_s":1.434191}1485gitlab # [ 268.486655] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.191Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b77041688a3518870fe49f64","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b77041688a3518870fe49f64: done: 0.014228 sec","job_status":"done","queue_duration_s":1.434175,"scheduling_latency_s":1.434191,"redis_calls":13,"redis_duration_s":0.001862,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000649,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001213,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009665,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014228,"completed_at":"2026-09-25T15:59:03.191Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1486gitlab # [ 268.528604] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.192Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f93251427fd8d5f63ae0c73d","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f93251427fd8d5f63ae0c73d: start","job_status":"start","queue_duration_s":1.449502,"scheduling_latency_s":1.449518}1487gitlab # [ 268.541448] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.206Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f93251427fd8d5f63ae0c73d","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f93251427fd8d5f63ae0c73d: done: 0.014394 sec","job_status":"done","queue_duration_s":1.449502,"scheduling_latency_s":1.449518,"redis_calls":13,"redis_duration_s":0.001825,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000613,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001212,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009918,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014394,"completed_at":"2026-09-25T15:59:03.206Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1488gitlab # [ 268.583243] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.207Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"22257ebc7b916bae929a5f9f","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-22257ebc7b916bae929a5f9f: start","job_status":"start","queue_duration_s":1.46508,"scheduling_latency_s":1.465095}1489gitlab # [ 268.596950] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.222Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"22257ebc7b916bae929a5f9f","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-22257ebc7b916bae929a5f9f: done: 0.014918 sec","job_status":"done","queue_duration_s":1.46508,"scheduling_latency_s":1.465095,"redis_calls":13,"redis_duration_s":0.001739,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000541,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001198,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010742,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014918,"completed_at":"2026-09-25T15:59:03.222Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1490gitlab # [ 268.637973] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.223Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"631e2f58e317d076e3aac428","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-631e2f58e317d076e3aac428: start","job_status":"start","queue_duration_s":1.481099,"scheduling_latency_s":1.481115}1491gitlab # [ 268.651401] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.239Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"631e2f58e317d076e3aac428","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-631e2f58e317d076e3aac428: done: 0.015718 sec","job_status":"done","queue_duration_s":1.481099,"scheduling_latency_s":1.481115,"redis_calls":13,"redis_duration_s":0.0014459999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000502,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000944,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009291,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015718,"completed_at":"2026-09-25T15:59:03.239Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1492gitlab # [ 268.692461] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.240Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9451cd81b9cd992ce60b28af","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9451cd81b9cd992ce60b28af: start","job_status":"start","queue_duration_s":1.497958,"scheduling_latency_s":1.497974}1493gitlab # [ 268.705794] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.255Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9451cd81b9cd992ce60b28af","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9451cd81b9cd992ce60b28af: done: 0.015379 sec","job_status":"done","queue_duration_s":1.497958,"scheduling_latency_s":1.497974,"redis_calls":13,"redis_duration_s":0.0018839999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000899,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000985,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010752,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015379,"completed_at":"2026-09-25T15:59:03.255Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1494gitlab # [ 268.746290] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.257Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a2ca881a75062fe161ea00a9","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a2ca881a75062fe161ea00a9: start","job_status":"start","queue_duration_s":1.514357,"scheduling_latency_s":1.514374}1495gitlab # [ 268.760596] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.271Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a2ca881a75062fe161ea00a9","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a2ca881a75062fe161ea00a9: done: 0.014536 sec","job_status":"done","queue_duration_s":1.514357,"scheduling_latency_s":1.514374,"redis_calls":13,"redis_duration_s":0.001445,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000494,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000951,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010387,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014536,"completed_at":"2026-09-25T15:59:03.271Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1496gitlab # [ 268.801607] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.272Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4d753ea7d786ba57cc9eaaeb","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4d753ea7d786ba57cc9eaaeb: start","job_status":"start","queue_duration_s":1.529929,"scheduling_latency_s":1.529946}1497gitlab # [ 268.814234] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.287Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4d753ea7d786ba57cc9eaaeb","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4d753ea7d786ba57cc9eaaeb: done: 0.014655 sec","job_status":"done","queue_duration_s":1.529929,"scheduling_latency_s":1.529946,"redis_calls":13,"redis_duration_s":0.0014529999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000516,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000937,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009318,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014655,"completed_at":"2026-09-25T15:59:03.287Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1498gitlab # [ 268.854133] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.289Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"106627ba38a522649730085f","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-106627ba38a522649730085f: start","job_status":"start","queue_duration_s":1.54681,"scheduling_latency_s":1.546827}1499gitlab # [ 268.867362] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.303Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"106627ba38a522649730085f","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-106627ba38a522649730085f: done: 0.014396 sec","job_status":"done","queue_duration_s":1.54681,"scheduling_latency_s":1.546827,"redis_calls":13,"redis_duration_s":0.0023439999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001475,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000869,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010176,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014396,"completed_at":"2026-09-25T15:59:03.303Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1500gitlab # [ 268.907547] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.304Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad10929578a4bbcd71b45e65","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad10929578a4bbcd71b45e65: start","job_status":"start","queue_duration_s":1.562255,"scheduling_latency_s":1.562271}1501gitlab # [ 268.920689] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.319Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad10929578a4bbcd71b45e65","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad10929578a4bbcd71b45e65: done: 0.014581 sec","job_status":"done","queue_duration_s":1.562255,"scheduling_latency_s":1.562271,"redis_calls":13,"redis_duration_s":0.001583,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001075,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010363,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014581,"completed_at":"2026-09-25T15:59:03.319Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1502gitlab # [ 268.962365] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.320Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8287ae1033f2803b52a4778e","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8287ae1033f2803b52a4778e: start","job_status":"start","queue_duration_s":1.577897,"scheduling_latency_s":1.577913}1503gitlab # [ 268.975743] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.334Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8287ae1033f2803b52a4778e","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8287ae1033f2803b52a4778e: done: 0.013889 sec","job_status":"done","queue_duration_s":1.577897,"scheduling_latency_s":1.577913,"redis_calls":13,"redis_duration_s":0.001482,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000943,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009774,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013889,"completed_at":"2026-09-25T15:59:03.334Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1504gitlab # [ 269.016138] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.335Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e607e1007af2fb9d7873b691","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e607e1007af2fb9d7873b691: start","job_status":"start","queue_duration_s":1.592902,"scheduling_latency_s":1.592919}1505gitlab # [ 269.028649] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.350Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e607e1007af2fb9d7873b691","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e607e1007af2fb9d7873b691: done: 0.015088 sec","job_status":"done","queue_duration_s":1.592902,"scheduling_latency_s":1.592919,"redis_calls":13,"redis_duration_s":0.001445,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000534,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000911,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009279,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015088,"completed_at":"2026-09-25T15:59:03.350Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1506gitlab # [ 269.069583] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.352Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"74da2aefbd32669693a87193","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-74da2aefbd32669693a87193: start","job_status":"start","queue_duration_s":1.609718,"scheduling_latency_s":1.609735}1507gitlab # [ 269.082293] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.366Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"74da2aefbd32669693a87193","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-74da2aefbd32669693a87193: done: 0.014015 sec","job_status":"done","queue_duration_s":1.609718,"scheduling_latency_s":1.609735,"redis_calls":13,"redis_duration_s":0.001407,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000495,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000912,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009856,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014015,"completed_at":"2026-09-25T15:59:03.366Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1508gitlab # [ 269.122213] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.367Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b0b9358c7d01557d935334b2","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b0b9358c7d01557d935334b2: start","job_status":"start","queue_duration_s":1.624809,"scheduling_latency_s":1.624825}1509gitlab # [ 269.135055] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.381Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b0b9358c7d01557d935334b2","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b0b9358c7d01557d935334b2: done: 0.014149 sec","job_status":"done","queue_duration_s":1.624809,"scheduling_latency_s":1.624825,"redis_calls":13,"redis_duration_s":0.001414,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00049,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000924,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010075,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014149,"completed_at":"2026-09-25T15:59:03.381Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1510gitlab # [ 269.177201] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.382Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"43bb610e1184c8d106ef07af","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-43bb610e1184c8d106ef07af: start","job_status":"start","queue_duration_s":1.640026,"scheduling_latency_s":1.640043}1511gitlab # [ 269.190042] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.396Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"43bb610e1184c8d106ef07af","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-43bb610e1184c8d106ef07af: done: 0.014097 sec","job_status":"done","queue_duration_s":1.640026,"scheduling_latency_s":1.640043,"redis_calls":13,"redis_duration_s":0.001491,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000516,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000975,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009297,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014097,"completed_at":"2026-09-25T15:59:03.396Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1512gitlab # [ 269.231022] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.398Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6da9f929758bbfe84ba0fe8f","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6da9f929758bbfe84ba0fe8f: start","job_status":"start","queue_duration_s":1.655906,"scheduling_latency_s":1.655924}1513gitlab # [ 269.243765] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.412Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6da9f929758bbfe84ba0fe8f","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6da9f929758bbfe84ba0fe8f: done: 0.014239 sec","job_status":"done","queue_duration_s":1.655906,"scheduling_latency_s":1.655924,"redis_calls":13,"redis_duration_s":0.0014320000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000506,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000926,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009968,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014239,"completed_at":"2026-09-25T15:59:03.412Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1514gitlab # [ 269.284045] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.413Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"698a44f7241f94df904a8ee9","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-698a44f7241f94df904a8ee9: start","job_status":"start","queue_duration_s":1.671206,"scheduling_latency_s":1.671223}1515gitlab # [ 269.297297] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.428Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"698a44f7241f94df904a8ee9","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-698a44f7241f94df904a8ee9: done: 0.014262 sec","job_status":"done","queue_duration_s":1.671206,"scheduling_latency_s":1.671223,"redis_calls":13,"redis_duration_s":0.00146,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000933,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010018,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014262,"completed_at":"2026-09-25T15:59:03.428Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1516gitlab # [ 269.337095] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.429Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0e3a6b0c1995a98f62fac4d1","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0e3a6b0c1995a98f62fac4d1: start","job_status":"start","queue_duration_s":1.686498,"scheduling_latency_s":1.686514}1517gitlab # [ 269.349884] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.443Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0e3a6b0c1995a98f62fac4d1","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0e3a6b0c1995a98f62fac4d1: done: 0.014663 sec","job_status":"done","queue_duration_s":1.686498,"scheduling_latency_s":1.686514,"redis_calls":13,"redis_duration_s":0.001392,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00051,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000882,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009862,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014663,"completed_at":"2026-09-25T15:59:03.443Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1518gitlab # [ 269.390953] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.444Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"89fb3db94cf92b56320b8746","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-89fb3db94cf92b56320b8746: start","job_status":"start","queue_duration_s":1.702293,"scheduling_latency_s":1.70231}1519gitlab # [ 269.404576] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.460Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"89fb3db94cf92b56320b8746","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-89fb3db94cf92b56320b8746: done: 0.015725 sec","job_status":"done","queue_duration_s":1.702293,"scheduling_latency_s":1.70231,"redis_calls":13,"redis_duration_s":0.001595,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000554,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001041,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010411,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015725,"completed_at":"2026-09-25T15:59:03.460Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1520gitlab # [ 269.444560] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.462Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d78d89f68cac7dca1795d367","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d78d89f68cac7dca1795d367: start","job_status":"start","queue_duration_s":1.719847,"scheduling_latency_s":1.719864}1521gitlab # [ 269.457164] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.476Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d78d89f68cac7dca1795d367","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d78d89f68cac7dca1795d367: done: 0.014162 sec","job_status":"done","queue_duration_s":1.719847,"scheduling_latency_s":1.719864,"redis_calls":13,"redis_duration_s":0.001435,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000909,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009882,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014162,"completed_at":"2026-09-25T15:59:03.476Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1522gitlab # [ 269.498617] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.477Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b70824b945293a020d81f62","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b70824b945293a020d81f62: start","job_status":"start","queue_duration_s":1.735017,"scheduling_latency_s":1.735034}1523gitlab # [ 269.511922] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.492Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b70824b945293a020d81f62","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b70824b945293a020d81f62: done: 0.014458 sec","job_status":"done","queue_duration_s":1.735017,"scheduling_latency_s":1.735034,"redis_calls":13,"redis_duration_s":0.001493,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00051,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000983,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01027,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014458,"completed_at":"2026-09-25T15:59:03.492Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1524gitlab # [ 269.551988] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.493Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c37544153e5862eec981b750","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c37544153e5862eec981b750: start","job_status":"start","queue_duration_s":1.75049,"scheduling_latency_s":1.750506}1525gitlab # [ 269.564614] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.508Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c37544153e5862eec981b750","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c37544153e5862eec981b750: done: 0.014908 sec","job_status":"done","queue_duration_s":1.75049,"scheduling_latency_s":1.750506,"redis_calls":13,"redis_duration_s":0.001885,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000512,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001373,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010263,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014908,"completed_at":"2026-09-25T15:59:03.508Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1526gitlab # [ 269.606380] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.509Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8337efe57bf391145175631","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8337efe57bf391145175631: start","job_status":"start","queue_duration_s":1.767131,"scheduling_latency_s":1.767149}1527gitlab # [ 269.619882] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.524Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8337efe57bf391145175631","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8337efe57bf391145175631: done: 0.014335 sec","job_status":"done","queue_duration_s":1.767131,"scheduling_latency_s":1.767149,"redis_calls":13,"redis_duration_s":0.001416,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000908,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009931,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014335,"completed_at":"2026-09-25T15:59:03.524Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1528gitlab # [ 269.660597] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.525Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1f890539b8274cd55df9564f","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1f890539b8274cd55df9564f: start","job_status":"start","queue_duration_s":1.782502,"scheduling_latency_s":1.782519}1529gitlab # [ 269.673586] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.539Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1f890539b8274cd55df9564f","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1f890539b8274cd55df9564f: done: 0.013993 sec","job_status":"done","queue_duration_s":1.782502,"scheduling_latency_s":1.782519,"redis_calls":13,"redis_duration_s":0.00149,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000965,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009796,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013993,"completed_at":"2026-09-25T15:59:03.539Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1530gitlab # [ 269.713308] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.540Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"354a57e68c942c5409c437ef","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-354a57e68c942c5409c437ef: start","job_status":"start","queue_duration_s":1.797567,"scheduling_latency_s":1.797583}1531gitlab # [ 269.725957] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.554Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"354a57e68c942c5409c437ef","created_at":"2026-09-25T15:59:01.349Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-75573214420a4807","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"39a7033ba0c9b74f28ff3355b95e22bc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.742Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-354a57e68c942c5409c437ef: done: 0.014498 sec","job_status":"done","queue_duration_s":1.797567,"scheduling_latency_s":1.797583,"redis_calls":13,"redis_duration_s":0.001761,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000502,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001259,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010147,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014498,"completed_at":"2026-09-25T15:59:03.554Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1532gitlab # [ 269.765802] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.555Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"a206a88998c91a618e0e778e","created_at":"2026-09-25T15:59:01.738Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bb44d84b485482d11d303b27e8293d24","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.745Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-a206a88998c91a618e0e778e: start","job_status":"start","queue_duration_s":1.810555,"scheduling_latency_s":1.810571}1533gitlab # [ 269.781731] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.589Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"a206a88998c91a618e0e778e","created_at":"2026-09-25T15:59:01.738Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-6d61844d21614755","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bb44d84b485482d11d303b27e8293d24","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.745Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-a206a88998c91a618e0e778e: done: 0.033653 sec","job_status":"done","queue_duration_s":1.810555,"scheduling_latency_s":1.810571,"redis_calls":10,"redis_duration_s":0.001461,"redis_read_bytes":135,"redis_write_bytes":1294,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000154,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":132,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000714,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000593,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021248,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00048418,"exclusive_lock_hold_duration_s":0.01949168,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.033653,"completed_at":"2026-09-25T15:59:03.589Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008563,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1534gitlab # [ 269.828438] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.590Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"8f38b01dc0e9961a7c3f432a","created_at":"2026-09-25T15:59:01.812Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-3ee01ab7b26f4cfc","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"53f6687ecfff1d73ff278f0aa3a59b3e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.825Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-8f38b01dc0e9961a7c3f432a: start","job_status":"start","queue_duration_s":1.765122,"scheduling_latency_s":1.765138}1535gitlab # [ 269.841648] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.602Z","message":"No existing merge request to be cleaned up."}1536gitlab # [ 269.843454] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.609Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"8f38b01dc0e9961a7c3f432a","created_at":"2026-09-25T15:59:01.812Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-3ee01ab7b26f4cfc","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"53f6687ecfff1d73ff278f0aa3a59b3e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.825Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-8f38b01dc0e9961a7c3f432a: done: 0.019265 sec","job_status":"done","queue_duration_s":1.765122,"scheduling_latency_s":1.765138,"redis_calls":11,"redis_duration_s":0.0015480000000000001,"redis_read_bytes":136,"redis_write_bytes":1131,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000146,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":109,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00062,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000782,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013602,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019265,"completed_at":"2026-09-25T15:59:03.609Z","load_balancing_strategy":"primary","db_duration_s":0.002146,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1537gitlab # [ 269.884621] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.611Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"fcd76eef743b70a60ca96a41","created_at":"2026-09-25T15:59:01.812Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-3ee01ab7b26f4cfc","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"53f6687ecfff1d73ff278f0aa3a59b3e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.825Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-fcd76eef743b70a60ca96a41: start","job_status":"start","queue_duration_s":1.786064,"scheduling_latency_s":1.786082}1538gitlab # [ 269.897604] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.625Z","message":"No existing merge request to be cleaned up."}1539gitlab # [ 269.899393] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.633Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"fcd76eef743b70a60ca96a41","created_at":"2026-09-25T15:59:01.812Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-3ee01ab7b26f4cfc","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"53f6687ecfff1d73ff278f0aa3a59b3e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.825Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-fcd76eef743b70a60ca96a41: done: 0.02172 sec","job_status":"done","queue_duration_s":1.786064,"scheduling_latency_s":1.786082,"redis_calls":10,"redis_duration_s":0.0014459999999999998,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000565,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000881,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013088,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02172,"completed_at":"2026-09-25T15:59:03.633Z","load_balancing_strategy":"primary","db_duration_s":0.002005,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1540gitlab # [ 269.938905] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.634Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0ba118ceea0240e848f273fe","created_at":"2026-09-25T15:59:01.812Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-3ee01ab7b26f4cfc","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"53f6687ecfff1d73ff278f0aa3a59b3e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.825Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0ba118ceea0240e848f273fe: start","job_status":"start","queue_duration_s":1.808866,"scheduling_latency_s":1.808883}1541gitlab # [ 269.951595] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.644Z","message":"No existing merge request to be cleaned up."}1542gitlab # [ 269.953420] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.652Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0ba118ceea0240e848f273fe","created_at":"2026-09-25T15:59:01.812Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-3ee01ab7b26f4cfc","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"53f6687ecfff1d73ff278f0aa3a59b3e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.825Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0ba118ceea0240e848f273fe: done: 0.017739 sec","job_status":"done","queue_duration_s":1.808866,"scheduling_latency_s":1.808883,"redis_calls":10,"redis_duration_s":0.001464,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000545,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000919,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012483,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017739,"completed_at":"2026-09-25T15:59:03.652Z","load_balancing_strategy":"primary","db_duration_s":0.00139,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1543gitlab # [ 269.993425] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.653Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d65caa0a816cd93258fa1b30","created_at":"2026-09-25T15:59:01.812Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-3ee01ab7b26f4cfc","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"53f6687ecfff1d73ff278f0aa3a59b3e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.825Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d65caa0a816cd93258fa1b30: start","job_status":"start","queue_duration_s":1.828102,"scheduling_latency_s":1.82812}1544gitlab # [ 270.005890] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.663Z","message":"No existing merge request to be cleaned up."}1545gitlab # [ 270.007599] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:03.671Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d65caa0a816cd93258fa1b30","created_at":"2026-09-25T15:59:01.812Z","trace_propagation_headers":{"sentry-trace":"c3fe9bf23cf849fa9a421df4f1d08f37-3ee01ab7b26f4cfc","baggage":"sentry-trace_id=c3fe9bf23cf849fa9a421df4f1d08f37,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"53f6687ecfff1d73ff278f0aa3a59b3e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:01.825Z","job_size_bytes":2,"pid":1864,"sidekiq_tid":"malc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d65caa0a816cd93258fa1b30: done: 0.017343 sec","job_status":"done","queue_duration_s":1.828102,"scheduling_latency_s":1.82812,"redis_calls":10,"redis_duration_s":0.0012850000000000001,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000541,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000744,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012177,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017343,"completed_at":"2026-09-25T15:59:03.670Z","load_balancing_strategy":"primary","db_duration_s":0.00144,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1546gitlab # [ 282.647034] postgres[3145]: [3145] ERROR: relation "table_without_models" does not exist1547gitlab # [ 282.647402] postgres[3145]: [3145] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1548gitlab # [ 288.341683] runuser[2813]: pam_unix(runuser:session): session closed for user gitlab1549gitlab # [ 288.342505] setup-gitlab-start[2797]: + systemctl restart gitlab-sidekiq.service1550gitlab # [ 288.373801] systemd[1]: Stopping gitlab-sidekiq.service...1551gitlab # [ 288.376211] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:24.266Z","message":"Shutting down"}1552gitlab # [ 288.378429] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:24.266Z","message":"Scheduler exiting..."}1553gitlab # [ 288.380764] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:24.267Z","message":"Terminating quiet threads for default capsule"}1554gitlab # [ 288.383379] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:24.267Z","message":"Scheduler exiting..."}1555gitlab # [ 288.877761] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:24.768Z","message":"Pausing to allow jobs to finish..."}1556gitlab # [ 293.381881] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:29.271Z","message":"Bye!"}1557gitlab # [ 293.415111] sidekiq-cluster[1864]: {"severity":"INFO","time":"2026-09-25T15:59:29.304Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":1864,"worker_id":"sidekiq_0","memwd_rss_bytes":925323264,"retry":0}1558gitlab # [ 293.877713] systemd[1]: gitlab-sidekiq.service: Deactivated successfully.1559gitlab # [ 293.879384] systemd[1]: Stopped gitlab-sidekiq.service.1560gitlab # [ 293.880753] systemd[1]: gitlab-sidekiq.service: Consumed 1min 43.633s CPU time over 3min 14.175s wall clock time, 944.1M memory peak, 5.3M read from disk.1561gitlab # [ 293.883968] systemd[1]: Started gitlab-sidekiq.service.1562gitlab # [ 293.885824] setup-gitlab-start[2797]: + mkdir -p /var/lib/secrets1563gitlab # [ 293.887288] setup-gitlab-start[2797]: + echo nixbot-test-token-of-20-chars1564gitlab # [ 293.888904] setup-gitlab-start[2797]: + chmod 644 /var/lib/secrets/gitlab-token1565gitlab # [ 293.890644] setup-gitlab-start[2797]: + timeout 300 bash -c $'until curl -fs -H "PRIVATE-TOKEN: $0" http://gitlab/api/v4/projects/root%2Ftest-flake > /dev/null; do\n curl -s -X POST -H "PRIVATE-TOKEN: $0" -H "Content-Type: application/json" \\\n -d "{\\"name\\":\\"test-flake\\",\\"visibility\\":\\"public\\",\\"topics\\":[\\"build-with-buildbot\\"]}" \\\n http://gitlab/api/v4/projects | jq -c "{id, message, error}"\n sleep 5\ndone' nixbot-test-token-of-20-chars1566gitlab # [ 294.231926] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:15:59:30 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 404 35 "" "curl/8.21.0" 3151567gitlab # [ 295.020294] gitaly[914]: time="2026-09-25T15:59:30.909Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M3CMP3TQRQPXKJJQWNJ4ESMF error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-25T15:59:40.008" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-25T15:59:30.909" pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1568gitlab # [ 295.020576] gitaly[914]: time="2026-09-25T15:59:30.910Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3CMP3TQRQPXKJJQWNJ4ESMF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-25T15:59:40.008" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-25T15:59:30.909" grpc.time_ms=0.431 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1569gitlab # [ 295.182318] gitaly[914]: time="2026-09-25T15:59:31.072Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M3CMP3TQRQPXKJJQWNJ4ESMF error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-25T15:59:41.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-25T15:59:31.072" pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1570gitlab # [ 295.199088] gitaly[914]: time="2026-09-25T15:59:31.073Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3CMP3TQRQPXKJJQWNJ4ESMF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-25T15:59:41.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-25T15:59:31.072" grpc.time_ms=0.325 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1571gitlab # [ 295.260938] gitaly[914]: time="2026-09-25T15:59:31.151Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3CMP3TQRQPXKJJQWNJ4ESMF diskcache=4f825026-4569-4a44-a96d-7121e66ac217 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-25T15:59:41.044" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-25T15:59:31.114" pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1572gitlab # [ 295.277939] gitaly[914]: time="2026-09-25T15:59:31.151Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1439 command.oublock=32 command.real_time_ms=23 command.spawn_token_fork_ms=0 command.system_time_ms=15 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMP3TQRQPXKJJQWNJ4ESMF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-25T15:59:41.044" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-25T15:59:31.114" grpc.time_ms=36.887 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1573gitlab # [ 295.303662] gitaly[914]: time="2026-09-25T15:59:31.164Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M3CMP3TQRQPXKJJQWNJ4ESMF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-25T15:59:41.093" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-25T15:59:31.164" pid=914 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1574gitlab # [ 295.321790] gitaly[914]: time="2026-09-25T15:59:31.165Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3CMP3TQRQPXKJJQWNJ4ESMF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-25T15:59:41.093" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-25T15:59:31.164" grpc.time_ms=0.716 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1575gitlab # [ 295.338515] gitaly[914]: time="2026-09-25T15:59:31.179Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMP3TQRQPXKJJQWNJ4ESMF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-25T15:59:41.099" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-25T15:59:31.170" grpc.time_ms=9.256 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1576gitlab # [ 295.363927] gitaly[914]: time="2026-09-25T15:59:31.254Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M3CMP3TQRQPXKJJQWNJ4ESMF error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-25T15:59:41.183" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-25T15:59:31.254" pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1577gitlab # [ 295.377710] gitaly[914]: time="2026-09-25T15:59:31.254Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3CMP3TQRQPXKJJQWNJ4ESMF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-25T15:59:41.183" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-25T15:59:31.254" grpc.time_ms=0.34 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1578gitlab # [ 295.394697] gitaly[914]: time="2026-09-25T15:59:31.285Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3CMP3TQRQPXKJJQWNJ4ESMF diskcache=3897caec-2491-466c-a529-5c240ff7a616 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-25T15:59:41.186" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-25T15:59:31.257" pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1579gitlab # [ 295.407841] gitaly[914]: time="2026-09-25T15:59:31.285Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1437 command.oublock=32 command.real_time_ms=16 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=7 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMP3TQRQPXKJJQWNJ4ESMF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-25T15:59:41.186" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-25T15:59:31.257" grpc.time_ms=28.494 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1580gitlab # [ 295.423277] gitaly[914]: time="2026-09-25T15:59:31.290Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M3CMP3TQRQPXKJJQWNJ4ESMF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-25T15:59:41.219" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-25T15:59:31.290" pid=914 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1581gitlab # [ 295.434462] gitaly[914]: time="2026-09-25T15:59:31.290Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3CMP3TQRQPXKJJQWNJ4ESMF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-25T15:59:41.219" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-25T15:59:31.290" grpc.time_ms=0.59 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1582gitlab # [ 295.445960] gitaly[914]: time="2026-09-25T15:59:31.302Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMP3TQRQPXKJJQWNJ4ESMF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-25T15:59:41.223" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-25T15:59:31.293" grpc.time_ms=8.543 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1583gitlab # [ 295.460069] gitaly[914]: time="2026-09-25T15:59:31.330Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3CMP3TQRQPXKJJQWNJ4ESMF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-25T15:59:41.260" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-25T15:59:31.330" grpc.time_ms=0.161 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1584gitlab # [ 295.471109] gitaly[914]: time="2026-09-25T15:59:31.338Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3CMP3TQRQPXKJJQWNJ4ESMF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-25T15:59:41.267" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-25T15:59:31.337" grpc.time_ms=0.283 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1585gitlab # [ 296.113204] gitaly[914]: time="2026-09-25T15:59:32.003Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMP3TQRQPXKJJQWNJ4ESMF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-25T15:59:41.092" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-25T15:59:31.993" grpc.time_ms=9.963 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1586gitlab # [ 296.693786] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:15:59:32 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 24481587gitlab # [ 296.697242] setup-gitlab-start[3221]: {"id":1,"message":null,"error":null}1588gitlab # [ 298.877726] sidekiq-cluster[3209]: {"severity":"INFO","time":"2026-09-25T15:59:34.731Z","message":"Starting cluster with 1 processes"}1589gitlab # [ 302.247213] gitaly[914]: time="2026-09-25T15:59:38.137Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPB48CRY1NXCNN3G65A3Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-25T15:59:48.007" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-25T15:59:38.128" grpc.time_ms=9.056 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1590gitlab # [ 304.367534] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:15:59:40 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 200 4924 "" "curl/8.21.0" 26491591gitlab # [ 304.371182] setup-gitlab-start[2797]: + mkdir -p /tmp/test-flake1592gitlab # [ 304.374008] setup-gitlab-start[2797]: + cd /tmp/test-flake1593gitlab # [ 304.375172] setup-gitlab-start[2797]: + git init -b master1594gitlab # [ 304.396867] setup-gitlab-start[3314]: Initialized empty Git repository in /tmp/test-flake/.git/1595gitlab # [ 304.398677] setup-gitlab-start[2797]: + git config user.name test1596gitlab # [ 304.401207] setup-gitlab-start[2797]: + git config user.email test@example.com1597gitlab # [ 304.405213] setup-gitlab-start[2797]: + cat1598gitlab # [ 304.408386] setup-gitlab-start[2797]: + git add flake.nix1599gitlab # [ 304.413522] setup-gitlab-start[2797]: + git commit -m 'initial commit'1600gitlab # [ 304.436778] setup-gitlab-start[3319]: [master (root-commit) 5a8336d] initial commit1601gitlab # [ 304.438502] setup-gitlab-start[3319]: 1 file changed, 10 insertions(+)1602gitlab # [ 304.439964] setup-gitlab-start[3319]: create mode 100644 flake.nix1603gitlab # [ 304.441876] setup-gitlab-start[2797]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git1604gitlab # [ 304.444516] setup-gitlab-start[2797]: + git push -u origin master1605gitlab # [ 304.518522] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:15:59:40 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 451606gitlab # [ 304.650771] gitaly[914]: time="2026-09-25T15:59:40.540Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3CMPDVT12S63TAWK0VA6GXC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T15:59:40.529" grpc.time_ms=10.736 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1607gitlab # [ 304.672595] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:15:59:40 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 1311608gitlab # [ 304.891747] gitaly[914]: time="2026-09-25T15:59:40.782Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPE07CPY3RA4RTWHTBTE3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-25T15:59:50.002" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-25T15:59:40.773" grpc.time_ms=9 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1609gitlab # [ 304.935796] gitaly[914]: time="2026-09-25T15:59:40.826Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=301 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3CMPE07CPY3RA4RTWHTBTE3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-25T16:00:10.016" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-25T15:59:40.816" grpc.time_ms=9.882 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1610gitlab # [ 304.959630] gitaly[914]: time="2026-09-25T15:59:40.850Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M3CMPE07CPY3RA4RTWHTBTE3 duration_ms=171ns method=POST pid=914 status=200 url="http://unix/api/v4/internal/allowed"1611gitlab # [ 304.993938] gitaly[914]: time="2026-09-25T15:59:40.884Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M3CMPE07CPY3RA4RTWHTBTE3 duration_ms=33ns method=POST pid=914 status=200 url="http://unix/api/v4/internal/pre_receive"1612gitlab # [ 304.999337] gitaly[914]: time="2026-09-25T15:59:40.889Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3CMPE07CPY3RA4RTWHTBTE3 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5316 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-25T15:59:40.677" grpc.time_ms=211.947 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1613gitlab # [ 305.028631] gitaly[914]: time="2026-09-25T15:59:40.919Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3CMPE07CPY3RA4RTWHTBTE3 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4786 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-25T15:59:40.918" grpc.time_ms=0.726 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1614gitlab # [ 305.076147] gitaly[914]: time="2026-09-25T15:59:40.966Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3CMPE07CPY3RA4RTWHTBTE3 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4785 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-25T15:59:40.966" grpc.time_ms=0.695 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1615gitlab # [ 305.107021] gitaly[914]: time="2026-09-25T15:59:40.997Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3CMPE07CPY3RA4RTWHTBTE3 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-25T15:59:40.997" grpc.time_ms=0.562 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1616gitlab # [ 305.207740] gitaly[914]: time="2026-09-25T15:59:41.098Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPE07CPY3RA4RTWHTBTE3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-25T15:59:51.018" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-25T15:59:41.088" grpc.time_ms=9.252 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1617gitlab # [ 305.236537] gitaly[914]: time="2026-09-25T15:59:41.127Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1434 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=0 command.system_time_ms=11 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPE07CPY3RA4RTWHTBTE3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-25T15:59:51.036" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-25T15:59:41.106" grpc.time_ms=20.031 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1618gitlab # [ 305.259401] gitaly[914]: time="2026-09-25T15:59:41.136Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M3CMPE07CPY3RA4RTWHTBTE3 duration_ms=111ns method=POST pid=914 status=200 url="http://unix/api/v4/internal/post_receive"1619gitlab # [ 305.264342] gitaly[914]: time="2026-09-25T15:59:41.137Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3CMPE07CPY3RA4RTWHTBTE3 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4810 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-25T15:59:41.024" grpc.time_ms=113.004 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1620gitlab # [ 305.280827] gitaly[914]: time="2026-09-25T15:59:41.149Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M3CMPE07CPY3RA4RTWHTBTE3 diskcache=5f3a51fe-3667-4a25-a806-bcb43fd0aa6b grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T15:59:40.619" pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1621gitlab # [ 305.295313] gitlab-workhorse[821]: time="2026-09-25T15:59:41Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521622gitlab # [ 305.297811] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:15:59:41 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 5981623gitlab # [ 305.300571] systemd[1]: Finished setup-gitlab.service.1624gitlab # [ 305.301629] setup-gitlab-start[3323]: To http://gitlab/root/test-flake.git1625gitlab # [ 305.302769] setup-gitlab-start[3323]: * [new branch] master -> master1626gitlab # [ 305.303891] setup-gitlab-start[3323]: branch 'master' set up to track 'origin/master'.1627gitlab # [ 305.305222] gitaly[914]: time="2026-09-25T15:59:41.149Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=188 command.inblock=8 command.majflt=0 command.maxrssanon_bytes=847872 command.minflt=13438 command.oublock=32 command.real_time_ms=523 command.spawn_token_fork_ms=0 command.system_time_ms=91 command.user_time_ms=97 component=gitaly.StreamServerInterceptor correlation_id=01M3CMPE07CPY3RA4RTWHTBTE3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T15:59:40.619" grpc.time_ms=529.303 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1628gitlab # [ 305.322266] systemd[1]: Starting nixbot CI service...1629gitlab: (finished: waiting for unit setup-gitlab.service, in 306.16 seconds)1630gitlab: waiting for unit nixbot.service1631gitlab # [ 306.449168] nixbot[3401]: {"ts": "2026-09-25T15:59:42.339Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1632gitlab # [ 306.477839] nixbot[3401]: {"ts": "2026-09-25T15:59:42.368Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1633gitlab # [ 306.499333] nixbot[3401]: {"ts": "2026-09-25T15:59:42.389Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1634gitlab # [ 306.503633] nixbot[3401]: {"ts": "2026-09-25T15:59:42.394Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1635gitlab # [ 306.507522] nixbot[3401]: {"ts": "2026-09-25T15:59:42.397Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1636gitlab # [ 306.512570] nixbot[3401]: {"ts": "2026-09-25T15:59:42.403Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1637gitlab # [ 306.516118] nixbot[3401]: {"ts": "2026-09-25T15:59:42.406Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1638gitlab # [ 306.519599] nixbot[3401]: {"ts": "2026-09-25T15:59:42.407Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1639gitlab # [ 306.523289] nixbot[3401]: {"ts": "2026-09-25T15:59:42.413Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1640gitlab # [ 306.526975] nixbot[3401]: {"ts": "2026-09-25T15:59:42.414Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1641gitlab # [ 306.531674] nixbot[3401]: {"ts": "2026-09-25T15:59:42.419Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1642gitlab # [ 306.535191] nixbot[3401]: {"ts": "2026-09-25T15:59:42.424Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1643gitlab # [ 306.538801] nixbot[3401]: {"ts": "2026-09-25T15:59:42.427Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1644gitlab # [ 306.542461] nixbot[3401]: {"ts": "2026-09-25T15:59:42.429Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1645gitlab # [ 306.546290] nixbot[3401]: {"ts": "2026-09-25T15:59:42.432Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1646gitlab # [ 306.549893] nixbot[3401]: {"ts": "2026-09-25T15:59:42.434Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1647gitlab # [ 306.553465] nixbot[3401]: {"ts": "2026-09-25T15:59:42.435Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1648gitlab # [ 306.557162] nixbot[3401]: {"ts": "2026-09-25T15:59:42.436Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1649gitlab # [ 306.560748] nixbot[3401]: {"ts": "2026-09-25T15:59:42.440Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"}1650gitlab # [ 306.564365] nixbot[3401]: {"ts": "2026-09-25T15:59:42.441Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"}1651gitlab # [ 306.567723] nixbot[3401]: {"ts": "2026-09-25T15:59:42.442Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"}1652gitlab # [ 306.571298] nixbot[3401]: {"ts": "2026-09-25T15:59:42.446Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"}1653gitlab # [ 306.575058] nixbot[3401]: {"ts": "2026-09-25T15:59:42.448Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"}1654gitlab # [ 306.578708] nixbot[3401]: {"ts": "2026-09-25T15:59:42.449Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"}1655gitlab # [ 306.582171] nixbot[3401]: {"ts": "2026-09-25T15:59:42.450Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"}1656gitlab # [ 306.585620] nixbot[3401]: {"ts": "2026-09-25T15:59:42.451Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"}1657gitlab # [ 306.589113] nixbot[3401]: {"ts": "2026-09-25T15:59:42.453Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"}1658gitlab # [ 306.592443] nixbot[3401]: {"ts": "2026-09-25T15:59:42.453Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"}1659gitlab # [ 306.595411] nixbot[3401]: {"ts": "2026-09-25T15:59:42.454Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"}1660gitlab # [ 306.598615] nixbot[3401]: {"ts": "2026-09-25T15:59:42.465Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"}1661gitlab # [ 306.601717] nixbot[3401]: {"ts": "2026-09-25T15:59:42.468Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"}1662gitlab # [ 306.604586] nixbot[3401]: {"ts": "2026-09-25T15:59:42.470Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"}1663gitlab # [ 306.607337] nixbot[3401]: {"ts": "2026-09-25T15:59:42.471Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 32, "migration": "upload_queue"}1664gitlab # [ 306.610057] nixbot[3401]: {"ts": "2026-09-25T15:59:42.474Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 33, "migration": "effect_runs_catchup"}1665gitlab # [ 306.615014] nixbot[3401]: {"ts": "2026-09-25T15:59:42.475Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 34, "migration": "public_badges"}1666gitlab # [ 306.622457] nixbot[3401]: {"ts": "2026-09-25T15:59:42.480Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 35, "migration": "settle_reset_effects"}1667gitlab # [ 306.625169] nixbot[3401]: {"ts": "2026-09-25T15:59:42.482Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 36, "migration": "pr_approvals"}1668gitlab # [ 307.005460] nixbot[3401]: INFO: Started server process [3401]1669gitlab # [ 307.007163] nixbot[3401]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)1670gitlab # [ 307.042113] nixbot[3401]: INFO: ::ffff:127.0.0.1:40612 - "GET /health HTTP/1.1" 200 OK1671gitlab # [ 307.048537] systemd[1]: Started nixbot CI service.1672gitlab # [ 307.049925] systemd[1]: Reached target Multi-User System.1673gitlab # [ 307.051174] systemd[1]: Startup finished in 711ms (kernel) + 3.642s (initrd) + 5min 2.696s (userspace) = 5min 7.050s.1674gitlab # [ 307.391781] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:15:59:43 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 3501675gitlab # [ 307.396131] nixbot[3401]: {"ts": "2026-09-25T15:59:43.283Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1676gitlab # [ 307.413137] nixbot[3401]: {"ts": "2026-09-25T15:59:43.303Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1677gitlab # [ 307.493524] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:15:59:43 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 711678gitlab # [ 307.497716] nixbot[3401]: {"ts": "2026-09-25T15:59:43.385Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1679gitlab # [ 307.501591] nixbot[3401]: {"ts": "2026-09-25T15:59:43.387Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1680gitlab # [ 307.648141] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:15:59:43 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 918 "" "python-httpx/0.28.1" 1451681gitlab # [ 307.651783] nixbot[3401]: {"ts": "2026-09-25T15:59:43.536Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""}1682gitlab: (finished: waiting for unit nixbot.service, in 2.08 seconds)1683gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1684gitlab # [ 307.828109] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:15:59:43 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 1611685gitlab # [ 307.830758] nixbot[3401]: {"ts": "2026-09-25T15:59:43.717Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1686gitlab # [ 307.838084] nixbot[3401]: INFO: ::ffff:127.0.0.1:40616 - "GET /health HTTP/1.1" 200 OK1687gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.04 seconds)1688subtest: project discovered and webhook registered1689gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1690gitlab # [ 307.904668] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:15:59:43 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "python-httpx/0.28.1" 591691gitlab # [ 307.908600] nixbot[3401]: {"ts": "2026-09-25T15:59:43.799Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1692gitlab # [ 307.981960] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:15:59:43 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 918 "" "python-httpx/0.28.1" 661693gitlab # [ 307.987657] nixbot[3401]: {"ts": "2026-09-25T15:59:43.876Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""}1694gitlab # [ 308.000116] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:15:59:43 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "curl/8.21.0" 1231695gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.16 seconds)1696(finished: subtest: project discovered and webhook registered, in 0.16 seconds)1697subtest: push delivers the webhook1698gitlab: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails'1699gitlab # [ 308.095851] gitaly[914]: time="2026-09-25T15:59:43.985Z" level=info msg="finished unary call with code OK" catfile.duration_ms=3 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=3 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=576 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPH8DMCTP4YPYS8AA6Z33 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-09-25T16:00:13.069" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-09-25T15:59:43.970" grpc.time_ms=15.165 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1700gitlab # [ 308.119757] gitaly[914]: time="2026-09-25T15:59:44.009Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPH8DMCTP4YPYS8AA6Z33 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-25T16:00:14.007" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T15:59:44.007" grpc.time_ms=1.142 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1701gitlab # [ 308.135505] gitaly[914]: time="2026-09-25T15:59:44.025Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=581 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPH8DMCTP4YPYS8AA6Z33 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-09-25T15:59:54.002" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T15:59:44.013" grpc.time_ms=11.332 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1702gitlab # [ 308.189699] redis[790]: 10 changes in 300 seconds. Saving...1703gitlab # [ 308.191424] redis[790]: Background saving started by pid 34611704gitlab # [ 308.195795] redis[3461]: BGSAVE done, 415 keys saved, 0 keys skipped, 107956 bytes written.1705gitlab # [ 308.199037] redis[3461]: DB saved on disk1706gitlab # [ 308.200461] redis[3461]: Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB1707gitlab # [ 308.290501] redis[790]: Background saving terminated with success1708gitlab # [ 308.314553] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:15:59:44 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 3191709gitlab # [ 308.318498] nixbot[3401]: {"ts": "2026-09-25T15:59:44.206Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""}1710gitlab # [ 308.413282] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:15:59:44 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 921711gitlab # [ 308.417463] nixbot[3401]: {"ts": "2026-09-25T15:59:44.304Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""}1712gitlab # [ 308.421911] nixbot[3401]: {"ts": "2026-09-25T15:59:44.310Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1713gitlab # [ 308.530683] gitaly[914]: time="2026-09-25T15:59:44.421Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=581 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01M3CMPHQ1G5XHTMYDNBTGJRDF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T15:59:44.409" grpc.time_ms=11.843 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc1714gitlab # [ 308.552051] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:15:59:44 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 691715gitlab # [ 308.661304] gitaly[914]: time="2026-09-25T15:59:44.551Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPHS8WEVG9QT51YW32D0Q grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T15:59:44.541" pid=914 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc1716gitlab # [ 308.678686] gitaly[914]: time="2026-09-25T15:59:44.552Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=7 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPHS8WEVG9QT51YW32D0Q grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T15:59:44.541" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc1717gitlab # [ 308.700462] gitlab-workhorse[821]: time="2026-09-25T15:59:44Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1131718gitlab # [ 308.703445] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:15:59:44 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 1441719gitlab # [ 308.706650] gitaly[914]: time="2026-09-25T15:59:44.552Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=7 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPHS8WEVG9QT51YW32D0Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T15:59:44.541" grpc.time_ms=10.747 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc1720gitlab # [ 308.771633] gitaly[914]: time="2026-09-25T15:59:44.662Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPHXW53ZJDXV6B5SZ6EVH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T15:59:44.644" pid=914 remote_ip="2001:db8:1::1" request_sha=8425b40be0c352e8d4c7135b1bb206db69faea9e response_bytes=223 span.kind=server system=grpc1721gitlab # [ 308.788861] gitaly[914]: time="2026-09-25T15:59:44.662Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1137 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=15 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPHXW53ZJDXV6B5SZ6EVH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T15:59:44.644" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc1722gitlab # [ 308.811631] gitlab-workhorse[821]: time="2026-09-25T15:59:44Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2231723gitlab # [ 308.814853] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:15:59:44 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 223 "" "git/2.55.0" 901724gitlab # [ 308.817909] gitaly[914]: time="2026-09-25T15:59:44.662Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1137 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=15 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPHXW53ZJDXV6B5SZ6EVH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T15:59:44.644" grpc.time_ms=18.026 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc1725gitlab # [ 308.876156] gitaly[914]: time="2026-09-25T15:59:44.766Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=577 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3CMPJ1Y4HTEVPF8DG1W997Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T15:59:44.757" grpc.time_ms=9.656 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc1726gitlab # [ 308.898141] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:15:59:44 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 641727gitlab # [ 308.943133] gitaly[914]: time="2026-09-25T15:59:44.833Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPJ40V9ZPMFWTH9ZSJMHE grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T15:59:44.822" pid=914 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1728gitlab # [ 308.960596] gitaly[914]: time="2026-09-25T15:59:44.833Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=587 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPJ40V9ZPMFWTH9ZSJMHE grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T15:59:44.822" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc1729gitlab # [ 308.981154] gitlab-workhorse[821]: time="2026-09-25T15:59:44Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671730gitlab # [ 308.983774] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:15:59:44 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 661731gitlab # [ 308.986500] gitaly[914]: time="2026-09-25T15:59:44.834Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=587 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPJ40V9ZPMFWTH9ZSJMHE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T15:59:44.822" grpc.time_ms=12.02 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc1732gitlab # [ 309.060947] gitaly[914]: time="2026-09-25T15:59:44.951Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=577 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3CMPJ7GADQSFCP97K7NG9M9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=192 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T15:59:44.940" grpc.time_ms=10.712 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc1733gitlab # [ 309.082109] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:15:59:44 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 711734gitlab # [ 309.137881] gitaly[914]: time="2026-09-25T15:59:45.027Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPJ9TA6F04RJ57888GZ62 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T15:59:45.009" pid=914 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc1735gitlab # [ 309.155467] gitaly[914]: time="2026-09-25T15:59:45.027Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1104 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=7 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPJ9TA6F04RJ57888GZ62 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T15:59:45.009" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc1736gitlab # [ 309.176865] gitlab-workhorse[821]: time="2026-09-25T15:59:45Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2091737gitlab # [ 309.179699] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:15:59:45 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 731738gitlab # [ 309.182581] gitaly[914]: time="2026-09-25T15:59:45.027Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1104 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=7 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPJ9TA6F04RJ57888GZ62 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T15:59:45.009" grpc.time_ms=18.302 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc1739gitlab # [ 309.240771] gitaly[914]: time="2026-09-25T15:59:45.131Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPJD8JSSAJ9FEB6R8ZMN3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-25T16:00:15.030" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T15:59:45.130" grpc.time_ms=0.722 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1740gitlab # [ 309.283166] gitaly[914]: time="2026-09-25T15:59:45.173Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=589 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3CMPJD8JSSAJ9FEB6R8ZMN3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-25T16:00:15.063" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-25T15:59:45.164" grpc.time_ms=9.035 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1741gitlab # [ 309.311448] gitaly[914]: time="2026-09-25T15:59:45.201Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=570 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3CMPJD8JSSAJ9FEB6R8ZMN3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-25T15:59:55.121" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-25T15:59:45.192" grpc.time_ms=9.683 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1742gitlab # [ 309.334751] gitaly[914]: time="2026-09-25T15:59:45.225Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3CMPJD8JSSAJ9FEB6R8ZMN3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-25T15:59:55.144" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-25T15:59:45.215" grpc.time_ms=9.915 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1743gitlab # [ 309.432883] gitaly[914]: time="2026-09-25T15:59:45.323Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPJD8JSSAJ9FEB6R8ZMN3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-25T15:59:55.252" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-25T15:59:45.322" grpc.time_ms=0.412 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1744gitlab # [ 309.453941] gitaly[914]: time="2026-09-25T15:59:45.344Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=569 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=4 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPJD8JSSAJ9FEB6R8ZMN3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-25T15:59:55.263" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-25T15:59:45.333" grpc.time_ms=10.612 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1745gitlab # [ 309.496916] gitaly[914]: time="2026-09-25T15:59:45.387Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3CMPJD8JSSAJ9FEB6R8ZMN3 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4154 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-25T15:59:45.386" grpc.time_ms=0.579 pid=914 span.kind=server system=grpc1746gitlab # [ 309.529203] gitaly[914]: time="2026-09-25T15:59:45.419Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3CMPJD8JSSAJ9FEB6R8ZMN3 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4156 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-25T15:59:45.419" grpc.time_ms=0.511 pid=914 span.kind=server system=grpc1747gitlab # [ 309.550334] gitaly[914]: time="2026-09-25T15:59:45.441Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPJD8JSSAJ9FEB6R8ZMN3 diskcache=8f6393df-acbb-4abd-bbd8-587c0ac6dc63 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-25T15:59:55.277" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-25T15:59:45.348" pid=914 ref=refs/keep-around/5a8336d76172e1ecc548fc24aea01729df24f8a5 remote_ip="2001:db8:1::1" revision=5a8336d76172e1ecc548fc24aea01729df24f8a5 span.kind=server system=grpc username=root1748gitlab # [ 309.569208] gitaly[914]: time="2026-09-25T15:59:45.441Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=73 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5342 command.oublock=8 command.real_time_ms=87 command.spawn_token_fork_ms=0 command.system_time_ms=40 command.user_time_ms=33 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPJD8JSSAJ9FEB6R8ZMN3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-25T15:59:55.277" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-25T15:59:45.348" grpc.time_ms=93.135 pid=914 ref=refs/keep-around/5a8336d76172e1ecc548fc24aea01729df24f8a5 remote_ip="2001:db8:1::1" revision=5a8336d76172e1ecc548fc24aea01729df24f8a5 span.kind=server system=grpc username=root1749gitlab # [ 309.590545] gitaly[914]: time="2026-09-25T15:59:45.451Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPJD8JSSAJ9FEB6R8ZMN3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-25T16:00:15.350" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T15:59:45.450" grpc.time_ms=0.454 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1750gitlab # [ 309.782525] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:15:59:45 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/5a8336d76172e1ecc548fc24aea01729df24f8a5 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 6071751gitlab # [ 309.787345] nixbot[3401]: {"ts": "2026-09-25T15:59:45.673Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/5a8336d76172e1ecc548fc24aea01729df24f8a5 \"HTTP/1.1 201 Created\""}1752gitlab # [ 309.794420] nixbot[3401]: {"ts": "2026-09-25T15:59:45.684Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-5a8336d76172-0"}1753gitlab # [ 309.842034] postgres[862]: [862] LOG: checkpoint starting: time1754gitlab # [ 309.998787] nixbot[3401]: {"ts": "2026-09-25T15:59:45.889Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-5a8336d76172-0", "argv0": "bwrap"}1755gitlab # [ 310.131535] systemd[1]: Started Nix Daemon.1756gitlab # [ 310.220174] nix-daemon[3564]: accepted connection from pid 3552, user nixbot1757gitlab # [ 310.584960] gitaly[914]: time="2026-09-25T15:59:46.474Z" level=info msg="burden monitor stats" pid=9141758gitlab # [ 315.085142] nix-daemon[3564]: accepted connection from pid 3607, user nixbot1759gitlab # [ 315.099678] nix-daemon[3564]: accepted connection from pid 3552, user nixbot1760gitlab # [ 319.457600] nix-daemon[3564]: accepted connection from pid 3552, user nixbot1761gitlab # [ 319.581192] gitaly[914]: time="2026-09-25T15:59:55.471Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPWG8X1GH6MBNG9MBT8E3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-25T16:00:25.369" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T15:59:55.470" grpc.time_ms=0.839 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1762gitlab # [ 319.719922] gitaly[914]: time="2026-09-25T15:59:55.610Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3CMPWG8X1GH6MBNG9MBT8E3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-25T16:00:05.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-25T15:59:55.600" grpc.time_ms=10.18 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1763gitlab # [ 319.807389] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:15:59:55 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/5a8336d76172e1ecc548fc24aea01729df24f8a5 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 2971764gitlab # [ 319.812423] nixbot[3401]: {"ts": "2026-09-25T15:59:55.699Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/5a8336d76172e1ecc548fc24aea01729df24f8a5 \"HTTP/1.1 201 Created\""}1765gitlab # [ 319.875848] gitaly[914]: time="2026-09-25T15:59:55.766Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPWSR2TFT28ZVR5B7WY8Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-25T16:00:25.664" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T15:59:55.765" grpc.time_ms=0.832 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1766gitlab # [ 319.918512] gitaly[914]: time="2026-09-25T15:59:55.809Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3CMPWSR2TFT28ZVR5B7WY8Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-25T16:00:05.208" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-25T15:59:55.798" grpc.time_ms=10.164 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1767gitlab # [ 319.987470] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:15:59:55 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/5a8336d76172e1ecc548fc24aea01729df24f8a5 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1721768gitlab # [ 319.992107] nixbot[3401]: {"ts": "2026-09-25T15:59:55.879Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/5a8336d76172e1ecc548fc24aea01729df24f8a5 \"HTTP/1.1 201 Created\""}1769gitlab # [ 320.068572] gitaly[914]: time="2026-09-25T15:59:55.959Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPWZMNH75PFT4703H0HWF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-25T16:00:25.157" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T15:59:55.958" grpc.time_ms=0.872 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1770gitlab # [ 320.121098] gitaly[914]: time="2026-09-25T15:59:56.011Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3CMPWZMNH75PFT4703H0HWF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-25T16:00:06.001" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-25T15:59:56.001" grpc.time_ms=10.047 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1771gitlab # [ 320.189061] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:15:59:56 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/5a8336d76172e1ecc548fc24aea01729df24f8a5 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1861772gitlab # [ 320.193366] nixbot[3401]: {"ts": "2026-09-25T15:59:56.081Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/5a8336d76172e1ecc548fc24aea01729df24f8a5 \"HTTP/1.1 201 Created\""}1773gitlab # [ 320.611978] gitaly[914]: time="2026-09-25T15:59:56.502Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3CMPXGJ9YSKRMRJJG2TXAJG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T15:59:56.491" grpc.time_ms=10.28 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc1774gitlab # [ 320.636113] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:15:59:56 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 921775gitlab # [ 320.705992] gitaly[914]: time="2026-09-25T15:59:56.596Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPXKGJPSRKWMJ90TVWDT5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T15:59:56.586" pid=914 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1776gitlab # [ 320.722297] gitaly[914]: time="2026-09-25T15:59:56.596Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPXKGJPSRKWMJ90TVWDT5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T15:59:56.586" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc1777gitlab # [ 320.744130] gitlab-workhorse[821]: time="2026-09-25T15:59:56Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671778gitlab # [ 320.747152] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:15:59:56 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 681779gitlab # [ 320.750166] gitaly[914]: time="2026-09-25T15:59:56.597Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPXKGJPSRKWMJ90TVWDT5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T15:59:56.586" grpc.time_ms=10.901 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc1780gitlab # [ 320.981824] gitaly[914]: time="2026-09-25T15:59:56.872Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3CMPXQCJ3GGGVWR77KMJ5SR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T15:59:56.862" grpc.time_ms=10.298 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc1781gitlab # [ 321.003997] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:15:59:56 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 2201782gitlab # [ 321.051371] gitaly[914]: time="2026-09-25T15:59:56.942Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPXYA1HNDH8XR6RKTR4H0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T15:59:56.931" pid=914 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc1783gitlab # [ 321.068472] gitaly[914]: time="2026-09-25T15:59:56.942Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPXYA1HNDH8XR6RKTR4H0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T15:59:56.931" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc1784gitlab # [ 321.089763] gitlab-workhorse[821]: time="2026-09-25T15:59:56Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671785gitlab # [ 321.092558] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:15:59:56 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 681786gitlab # [ 321.095204] gitaly[914]: time="2026-09-25T15:59:56.942Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPXYA1HNDH8XR6RKTR4H0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T15:59:56.931" grpc.time_ms=11.308 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc1787gitlab # [ 324.431923] systemd[1]: Starting Logrotate Service...1788gitlab # [ 324.527068] systemd[1]: logrotate.service: Deactivated successfully.1789gitlab # [ 324.528748] systemd[1]: Finished Logrotate Service.1790gitlab # [ 339.630739] sidekiq-cluster[3274]: 2026-09-25T16:00:15.520Z pid=3274 tid=yg2 INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>3274, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>433438720}1791gitlab # [ 339.919477] sidekiq-cluster[3274]: W, [2026-09-25T16:00:15.809987 #3274] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1792gitlab # [ 339.984473] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:00:15.875Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1793gitlab # [ 343.639484] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:00:19.529Z","message":"GitLab reliable fetch activated!"}1794gitlab # [ 343.639702] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:00:19.529Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1795gitlab # [ 380.145520] postgres[4162]: [4162] ERROR: relation "table_without_models" does not exist1796gitlab # [ 380.145901] postgres[4162]: [4162] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1797gitlab # [ 385.019308] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:00.909Z","message":"Booted Rails 7.2.3.1 application in production environment"}1798gitlab # [ 385.024508] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:00.915Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1799gitlab # [ 385.027850] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:00.915Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1800gitlab # [ 385.030966] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:00.915Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1801gitlab # [ 385.034360] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:00.916Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1802gitlab: (finished: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails', in 77.62 seconds)1803gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1804gitlab # [ 385.766728] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:16:01:01 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 891805gitlab # [ 385.886475] gitaly[914]: time="2026-09-25T16:01:01.776Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M3CMRX6SQBBMXNM6A1FC7VZN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T16:01:01.765" grpc.time_ms=10.616 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1806gitlab # Enumerating objects: 5, done.1807gitlab # [ 385.908470] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:16:01:01 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 201 "" "git/2.55.0" 1191808gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1809gitlab # Delta compression using up to 4 threads1810gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1811gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 273 bytes | 273.00 KiB/s, done.1812gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1813gitlab # [ 386.206277] gitaly[914]: time="2026-09-25T16:01:02.096Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=302 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3CMRXBM0SVV3D8GDBMBN65N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-25T16:01:32.087" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-25T16:01:02.087" grpc.time_ms=8.883 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1814gitlab # [ 386.247977] gitaly[914]: time="2026-09-25T16:01:02.138Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M3CMRXBM0SVV3D8GDBMBN65N duration_ms=142ns method=POST pid=914 status=200 url="http://unix/api/v4/internal/allowed"1815gitlab # [ 386.313278] gitaly[914]: time="2026-09-25T16:01:02.203Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M3CMRXBM0SVV3D8GDBMBN65N duration_ms=64ns method=POST pid=914 status=200 url="http://unix/api/v4/internal/pre_receive"1816gitlab # [ 386.324656] gitaly[914]: time="2026-09-25T16:01:02.204Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3CMRXBM0SVV3D8GDBMBN65N grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5316 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-25T16:01:01.995" grpc.time_ms=209.121 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1817gitlab # [ 386.363873] gitaly[914]: time="2026-09-25T16:01:02.254Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3CMRXBM0SVV3D8GDBMBN65N grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4786 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-25T16:01:02.253" grpc.time_ms=1.108 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1818gitlab # [ 386.406027] gitaly[914]: time="2026-09-25T16:01:02.296Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3CMRXBM0SVV3D8GDBMBN65N grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4785 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-25T16:01:02.294" grpc.time_ms=1.715 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1819gitlab # [ 386.458266] gitaly[914]: time="2026-09-25T16:01:02.348Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3CMRXBM0SVV3D8GDBMBN65N grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-25T16:01:02.348" grpc.time_ms=0.646 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1820gitlab # [ 386.638601] gitaly[914]: time="2026-09-25T16:01:02.529Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M3CMRXBM0SVV3D8GDBMBN65N duration_ms=127ns method=POST pid=914 status=200 url="http://unix/api/v4/internal/post_receive"1821gitlab # [ 386.646083] gitaly[914]: time="2026-09-25T16:01:02.529Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3CMRXBM0SVV3D8GDBMBN65N grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4810 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-25T16:01:02.400" grpc.time_ms=128.796 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1822gitlab # [ 386.667701] gitaly[914]: time="2026-09-25T16:01:02.558Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M3CMRXBM0SVV3D8GDBMBN65N diskcache=3e17d7d1-cc3f-49b2-88b5-b677ea1651fb grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T16:01:01.935" pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1823gitlab # To http://gitlab/root/test-flake.git1824gitlab # [ 386.684141] gitaly[914]: time="2026-09-25T16:01:02.558Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=245 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=847872 command.minflt=13637 command.oublock=32 command.real_time_ms=617 command.spawn_token_fork_ms=0 command.system_time_ms=144 command.user_time_ms=101 component=gitaly.StreamServerInterceptor correlation_id=01M3CMRXBM0SVV3D8GDBMBN65N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=605 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T16:01:01.935" grpc.time_ms=623.225 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1825gitlab # 5a8336d..53d1e52 [ 386.705322] gitlab-workhorse[821]: time="2026-09-25T16:01:02Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521826gitlab # master -> master[ 386.708219] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:16:01:02 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 7461827gitlab # 1828gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.09 seconds)1829gitlab: must succeed: git -C /tmp/test-flake rev-parse master1830gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds)1831gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=53d1e523bb688b08820df8ef6496a15b07ee0ccf'1832gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=53d1e523bb688b08820df8ef6496a15b07ee0ccf', in 0.10 seconds)1833gitlab # [ 386.829661] nixbot[3401]: INFO: ::ffff:127.0.0.1:45974 - "GET /api/repos/gitlab/root/test-flake/builds?commit=53d1e523bb688b08820df8ef6496a15b07ee0ccf HTTP/1.1" 200 OK1834gitlab # [ 387.739657] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:03.629Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"b38fe9f3d2a6748bbac7eba8","created_at":"2026-09-25T15:59:31.193Z","trace_propagation_headers":{"sentry-trace":"51bf635a7348403bb2be85d6e4e84175-8fa4980fae43489e","baggage":"sentry-trace_id=51bf635a7348403bb2be85d6e4e84175,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3CMP3TQRQPXKJJQWNJ4ESMF","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:31.232Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-b38fe9f3d2a6748bbac7eba8: start","job_status":"start","queue_duration_s":92.396976,"scheduling_latency_s":92.397044}1835gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=53d1e523bb688b08820df8ef6496a15b07ee0ccf'1836gitlab # [ 387.862781] nixbot[3401]: INFO: ::ffff:127.0.0.1:45990 - "GET /api/repos/gitlab/root/test-flake/builds?commit=53d1e523bb688b08820df8ef6496a15b07ee0ccf HTTP/1.1" 200 OK1837gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=53d1e523bb688b08820df8ef6496a15b07ee0ccf', in 0.03 seconds)1838gitlab # [ 387.971816] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:03.861Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"b38fe9f3d2a6748bbac7eba8","created_at":"2026-09-25T15:59:31.193Z","trace_propagation_headers":{"sentry-trace":"51bf635a7348403bb2be85d6e4e84175-8fa4980fae43489e","baggage":"sentry-trace_id=51bf635a7348403bb2be85d6e4e84175,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3CMP3TQRQPXKJJQWNJ4ESMF","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:31.232Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-b38fe9f3d2a6748bbac7eba8: done: 0.233024 sec","job_status":"done","queue_duration_s":92.396976,"scheduling_latency_s":92.397044,"redis_calls":23,"redis_duration_s":0.005916,"redis_read_bytes":910,"redis_write_bytes":2540,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.003249,"redis_feature_flag_read_bytes":902,"redis_feature_flag_write_bytes":1375,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00155,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001117,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.121148,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.12230604,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.233024,"completed_at":"2026-09-25T16:01:03.861Z","load_balancing_strategy":"primary","db_duration_s":0.061274,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1839gitlab # [ 388.043618] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:03.863Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"7398376b4a36a643ff08cf0e","created_at":"2026-09-25T15:59:31.243Z","trace_propagation_headers":{"sentry-trace":"51bf635a7348403bb2be85d6e4e84175-8fa4980fae43489e","baggage":"sentry-trace_id=51bf635a7348403bb2be85d6e4e84175,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3CMP3TQRQPXKJJQWNJ4ESMF","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:31.249Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-7398376b4a36a643ff08cf0e: start","job_status":"start","queue_duration_s":92.614385,"scheduling_latency_s":92.61441}1840gitlab # [ 388.089643] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:03.979Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"7398376b4a36a643ff08cf0e","created_at":"2026-09-25T15:59:31.243Z","trace_propagation_headers":{"sentry-trace":"51bf635a7348403bb2be85d6e4e84175-8fa4980fae43489e","baggage":"sentry-trace_id=51bf635a7348403bb2be85d6e4e84175,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3CMP3TQRQPXKJJQWNJ4ESMF","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:31.249Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-7398376b4a36a643ff08cf0e: done: 0.116143 sec","job_status":"done","queue_duration_s":92.614385,"scheduling_latency_s":92.61441,"redis_calls":14,"redis_duration_s":0.00315,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000604,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001598,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000948,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.088907,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0885889,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.116143,"completed_at":"2026-09-25T16:01:03.979Z","load_balancing_strategy":"primary","db_duration_s":0.020517,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1841gitlab # [ 388.155059] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:03.981Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"ee045665d602a785668d521e","created_at":"2026-09-25T15:59:31.886Z","trace_propagation_headers":{"sentry-trace":"51bf635a7348403bb2be85d6e4e84175-8fa4980fae43489e","baggage":"sentry-trace_id=51bf635a7348403bb2be85d6e4e84175,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3CMP3TQRQPXKJJQWNJ4ESMF","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:31.893Z","job_size_bytes":5,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-ee045665d602a785668d521e: start","job_status":"start","queue_duration_s":92.087927,"scheduling_latency_s":92.087948}1842gitlab # [ 388.253183] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:04.143Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"ee045665d602a785668d521e","created_at":"2026-09-25T15:59:31.886Z","trace_propagation_headers":{"sentry-trace":"51bf635a7348403bb2be85d6e4e84175-8fa4980fae43489e","baggage":"sentry-trace_id=51bf635a7348403bb2be85d6e4e84175,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3CMP3TQRQPXKJJQWNJ4ESMF","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:31.893Z","job_size_bytes":5,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-ee045665d602a785668d521e: done: 0.16201 sec","job_status":"done","queue_duration_s":92.087927,"scheduling_latency_s":92.087948,"redis_calls":12,"redis_duration_s":0.002274,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000579,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001372,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000323,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.132293,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025408,"duration_s":0.16201,"completed_at":"2026-09-25T16:01:04.143Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.02445,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1843gitlab # [ 388.322442] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:04.144Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"3e6d9e0edadc73dac2202351","created_at":"2026-09-25T15:59:31.923Z","trace_propagation_headers":{"sentry-trace":"51bf635a7348403bb2be85d6e4e84175-8fa4980fae43489e","baggage":"sentry-trace_id=51bf635a7348403bb2be85d6e4e84175,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3CMP3TQRQPXKJJQWNJ4ESMF","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:31.928Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-3e6d9e0edadc73dac2202351: start","job_status":"start","queue_duration_s":92.216083,"scheduling_latency_s":92.216099}1844gitlab # [ 388.452467] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:04.342Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"3e6d9e0edadc73dac2202351","created_at":"2026-09-25T15:59:31.923Z","trace_propagation_headers":{"sentry-trace":"51bf635a7348403bb2be85d6e4e84175-8fa4980fae43489e","baggage":"sentry-trace_id=51bf635a7348403bb2be85d6e4e84175,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3CMP3TQRQPXKJJQWNJ4ESMF","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:31.928Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-3e6d9e0edadc73dac2202351: done: 0.197884 sec","job_status":"done","queue_duration_s":92.216083,"scheduling_latency_s":92.216099,"redis_calls":11,"redis_duration_s":0.001969,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000542,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001054,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000373,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.164,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.164,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.140631,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00028356,"duration_s":0.197884,"completed_at":"2026-09-25T16:01:04.342Z","load_balancing_strategy":"primary","db_duration_s":0.060661,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1845gitlab # [ 388.521294] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:04.344Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"b57b12a0702b9f93156df8ae","created_at":"2026-09-25T15:59:31.932Z","trace_propagation_headers":{"sentry-trace":"51bf635a7348403bb2be85d6e4e84175-8fa4980fae43489e","baggage":"sentry-trace_id=51bf635a7348403bb2be85d6e4e84175,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3CMP3TQRQPXKJJQWNJ4ESMF","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:31.937Z","job_size_bytes":89,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-b57b12a0702b9f93156df8ae: start","job_status":"start","queue_duration_s":92.406553,"scheduling_latency_s":92.406571}1846gitlab # [ 388.544279] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:04.380Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"b57b12a0702b9f93156df8ae","created_at":"2026-09-25T15:59:31.932Z","trace_propagation_headers":{"sentry-trace":"51bf635a7348403bb2be85d6e4e84175-8fa4980fae43489e","baggage":"sentry-trace_id=51bf635a7348403bb2be85d6e4e84175,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3CMP3TQRQPXKJJQWNJ4ESMF","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:31.937Z","job_size_bytes":89,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-b57b12a0702b9f93156df8ae: done: 0.035992 sec","job_status":"done","queue_duration_s":92.406553,"scheduling_latency_s":92.406571,"redis_calls":11,"redis_duration_s":0.001931,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000531,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001035,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000365,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02476,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00024822,"duration_s":0.035992,"completed_at":"2026-09-25T16:01:04.380Z","load_balancing_strategy":"primary","db_duration_s":0.004973,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1847gitlab # [ 388.592331] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:04.381Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 5a8336d76172e1ecc548fc24aea01729df24f8a5 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"70f9cfefed56268565e79180","created_at":"2026-09-25T15:59:41.064Z","trace_propagation_headers":{"sentry-trace":"1889fbea594747379be5acfaf38873cf-9254a39cb86c4afb","baggage":"sentry-trace_id=1889fbea594747379be5acfaf38873cf,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3CMPE07CPY3RA4RTWHTBTE3","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:41.070Z","job_size_bytes":185,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-70f9cfefed56268565e79180: start","job_status":"start","queue_duration_s":83.310861,"scheduling_latency_s":83.310878}1848gitlab # [ 388.610639] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:04.397Z","message":"changes: \"0000000000000000000000000000000000000000 5a8336d76172e1ecc548fc24aea01729df24f8a5 refs/heads/master\\n\""}1849gitlab # [ 388.613621] gitaly[914]: time="2026-09-25T16:01:04.432Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3CMPE07CPY3RA4RTWHTBTE3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-25T16:01:14.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-25T16:01:04.422" grpc.time_ms=10.025 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1850gitlab # [ 388.627270] gitaly[914]: time="2026-09-25T16:01:04.446Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPE07CPY3RA4RTWHTBTE3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-25T16:01:14.018" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-25T16:01:04.438" grpc.time_ms=8.1 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1851gitlab # [ 388.640873] gitaly[914]: time="2026-09-25T16:01:04.512Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M3CMPE07CPY3RA4RTWHTBTE3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-25T16:01:14.081" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-25T16:01:04.501" grpc.time_ms=10.613 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1852gitlab # [ 388.665346] gitaly[914]: time="2026-09-25T16:01:04.556Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3CMPE07CPY3RA4RTWHTBTE3 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3937 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-25T16:01:04.555" grpc.time_ms=0.453 pid=914 span.kind=server system=grpc1853gitlab # [ 388.694621] gitaly[914]: time="2026-09-25T16:01:04.585Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3CMPE07CPY3RA4RTWHTBTE3 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3939 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-25T16:01:04.584" grpc.time_ms=0.404 pid=914 span.kind=server system=grpc1854gitlab # [ 388.707648] gitaly[914]: time="2026-09-25T16:01:04.593Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPE07CPY3RA4RTWHTBTE3 diskcache=825d0350-9171-4a25-8a26-13d59501d315 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-25T16:01:14.103" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-25T16:01:04.524" pid=914 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1855gitlab # [ 388.720611] gitaly[914]: time="2026-09-25T16:01:04.593Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=71 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4866 command.oublock=8 command.real_time_ms=64 command.spawn_token_fork_ms=0 command.system_time_ms=33 command.user_time_ms=38 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPE07CPY3RA4RTWHTBTE3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-25T16:01:14.103" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-25T16:01:04.524" grpc.time_ms=68.967 pid=914 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1856gitlab # [ 388.736699] gitaly[914]: time="2026-09-25T16:01:04.609Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPE07CPY3RA4RTWHTBTE3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-25T16:01:14.188" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-25T16:01:04.609" grpc.time_ms=0.173 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1857gitlab # [ 388.747855] gitaly[914]: time="2026-09-25T16:01:04.626Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=860 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPE07CPY3RA4RTWHTBTE3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-25T16:01:14.193" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-25T16:01:04.614" grpc.time_ms=12.188 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1858gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=53d1e523bb688b08820df8ef6496a15b07ee0ccf'1859gitlab # [ 388.848223] gitaly[914]: time="2026-09-25T16:01:04.738Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=312 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3CMPE07CPY3RA4RTWHTBTE3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-09-25T16:01:34.032" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:04.732" grpc.time_ms=6.45 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1860gitlab # [ 388.887132] gitaly[914]: time="2026-09-25T16:01:04.777Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPE07CPY3RA4RTWHTBTE3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-25T16:01:14.007" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-25T16:01:04.777" grpc.time_ms=0.314 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1861gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=53d1e523bb688b08820df8ef6496a15b07ee0ccf', in 0.04 seconds)1862gitlab # [ 388.905248] gitaly[914]: time="2026-09-25T16:01:04.788Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=317 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3CMPE07CPY3RA4RTWHTBTE3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-25T16:01:14.010" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-09-25T16:01:04.781" grpc.time_ms=6.738 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1863gitlab # [ 388.927600] nixbot[3401]: INFO: ::ffff:127.0.0.1:46000 - "GET /api/repos/gitlab/root/test-flake/builds?commit=53d1e523bb688b08820df8ef6496a15b07ee0ccf HTTP/1.1" 200 OK1864gitlab # [ 388.930715] gitaly[914]: time="2026-09-25T16:01:04.814Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M3CMPE07CPY3RA4RTWHTBTE3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-25T16:01:14.035" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-25T16:01:04.805" grpc.time_ms=9.156 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1865gitlab # [ 388.948122] gitaly[914]: time="2026-09-25T16:01:04.831Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M3CMPE07CPY3RA4RTWHTBTE3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-25T16:01:14.051" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-25T16:01:04.821" grpc.time_ms=9.164 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1866gitlab # [ 388.970148] gitaly[914]: time="2026-09-25T16:01:04.860Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=882 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3CMPE07CPY3RA4RTWHTBTE3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-25T16:01:34.145" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:04.845" grpc.time_ms=14.994 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1867gitlab # [ 389.505426] gitaly[914]: time="2026-09-25T16:01:05.396Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3CMPE07CPY3RA4RTWHTBTE3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-25T16:01:15.004" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:05.394" grpc.time_ms=1.541 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1868gitlab # [ 389.536222] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:05.425Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 5a8336d76172e1ecc548fc24aea01729df24f8a5 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"70f9cfefed56268565e79180","created_at":"2026-09-25T15:59:41.064Z","trace_propagation_headers":{"sentry-trace":"1889fbea594747379be5acfaf38873cf-9254a39cb86c4afb","baggage":"sentry-trace_id=1889fbea594747379be5acfaf38873cf,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3CMPE07CPY3RA4RTWHTBTE3","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-25T15:59:41.070Z","job_size_bytes":185,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-70f9cfefed56268565e79180: done: 1.044106 sec","job_status":"done","queue_duration_s":83.310861,"scheduling_latency_s":83.310878,"gitaly_calls":13,"gitaly_duration_s":0.195974,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.039126,"redis_read_bytes":2103,"redis_write_bytes":21750,"redis_cache_calls":15,"redis_cache_duration_s":0.003653,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.006312,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.001416,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7838,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.00367,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.014814,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.009261,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.052,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.277,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.309,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.052,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.770178,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00057992,"exclusive_lock_hold_duration_s":0.02761828,"duration_s":1.044106,"completed_at":"2026-09-25T16:01:05.425Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.091689,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1869gitlab # [ 389.607905] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:05.427Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"ab64361f07a9df7f49cccceb","created_at":"2026-09-25T15:59:45.455Z","trace_propagation_headers":{"sentry-trace":"548c50fbf76b48c3a77abf32d111e9bd-2d24597bc98e41d4","baggage":"sentry-trace_id=548c50fbf76b48c3a77abf32d111e9bd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3CMPJD8JSSAJ9FEB6R8ZMN3","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:45.492Z","job_size_bytes":65,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-ab64361f07a9df7f49cccceb: start","job_status":"start","queue_duration_s":79.934352,"scheduling_latency_s":79.934372}1870gitlab # [ 389.625570] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:05.510Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"ab64361f07a9df7f49cccceb","created_at":"2026-09-25T15:59:45.455Z","trace_propagation_headers":{"sentry-trace":"548c50fbf76b48c3a77abf32d111e9bd-2d24597bc98e41d4","baggage":"sentry-trace_id=548c50fbf76b48c3a77abf32d111e9bd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3CMPJD8JSSAJ9FEB6R8ZMN3","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:45.492Z","job_size_bytes":65,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-ab64361f07a9df7f49cccceb: done: 0.083661 sec","job_status":"done","queue_duration_s":79.934352,"scheduling_latency_s":79.934372,"redis_calls":16,"redis_duration_s":0.003465,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.00189,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001227,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000348,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.057383,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026448,"duration_s":0.083661,"completed_at":"2026-09-25T16:01:05.510Z","load_balancing_strategy":"primary","db_duration_s":0.020681,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1871gitlab # [ 389.672174] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:05.512Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"615854da5f79e19264e4c897","created_at":"2026-09-25T15:59:45.494Z","trace_propagation_headers":{"sentry-trace":"548c50fbf76b48c3a77abf32d111e9bd-2d24597bc98e41d4","baggage":"sentry-trace_id=548c50fbf76b48c3a77abf32d111e9bd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3CMPJD8JSSAJ9FEB6R8ZMN3","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:45.501Z","job_size_bytes":65,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-615854da5f79e19264e4c897: start","job_status":"start","queue_duration_s":80.011006,"scheduling_latency_s":80.011022}1872gitlab # [ 389.721802] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:05.611Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"615854da5f79e19264e4c897","created_at":"2026-09-25T15:59:45.494Z","trace_propagation_headers":{"sentry-trace":"548c50fbf76b48c3a77abf32d111e9bd-2d24597bc98e41d4","baggage":"sentry-trace_id=548c50fbf76b48c3a77abf32d111e9bd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3CMPJD8JSSAJ9FEB6R8ZMN3","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:45.501Z","job_size_bytes":65,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-615854da5f79e19264e4c897: done: 0.099534 sec","job_status":"done","queue_duration_s":80.011006,"scheduling_latency_s":80.011022,"redis_calls":21,"redis_duration_s":0.004292000000000001,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.001757,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001347,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001188,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.064464,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00045768,"duration_s":0.099534,"completed_at":"2026-09-25T16:01:05.611Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.027169,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1873gitlab # [ 389.773995] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:05.613Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"00f61b4985cb1bc95e5766a3","created_at":"2026-09-25T15:59:45.647Z","trace_propagation_headers":{"sentry-trace":"548c50fbf76b48c3a77abf32d111e9bd-2d24597bc98e41d4","baggage":"sentry-trace_id=548c50fbf76b48c3a77abf32d111e9bd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3CMPJD8JSSAJ9FEB6R8ZMN3","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:45.653Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-00f61b4985cb1bc95e5766a3: start","job_status":"start","queue_duration_s":79.96014,"scheduling_latency_s":79.960164}1874gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=53d1e523bb688b08820df8ef6496a15b07ee0ccf'1875gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=53d1e523bb688b08820df8ef6496a15b07ee0ccf', in 0.03 seconds)1876gitlab # [ 389.941775] nixbot[3401]: INFO: ::ffff:127.0.0.1:46002 - "GET /api/repos/gitlab/root/test-flake/builds?commit=53d1e523bb688b08820df8ef6496a15b07ee0ccf HTTP/1.1" 200 OK1877gitlab # [ 390.073061] gitaly[914]: time="2026-09-25T16:01:05.963Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3CMPJD8JSSAJ9FEB6R8ZMN3 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-25T16:01:05.962" grpc.time_ms=0.609 pid=914 span.kind=server system=grpc1878gitlab # [ 390.103152] gitaly[914]: time="2026-09-25T16:01:05.993Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3CMPJD8JSSAJ9FEB6R8ZMN3 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-25T16:01:05.993" grpc.time_ms=0.659 pid=914 span.kind=server system=grpc1879gitlab # [ 390.133976] gitaly[914]: time="2026-09-25T16:01:06.024Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3CMPJD8JSSAJ9FEB6R8ZMN3 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-25T16:01:06.024" grpc.time_ms=0.547 pid=914 span.kind=server system=grpc1880gitlab # [ 390.150252] gitaly[914]: time="2026-09-25T16:01:06.033Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPJD8JSSAJ9FEB6R8ZMN3 diskcache=bc195f75-b25d-4b38-b6db-4706021a646a grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-25T16:01:35.228" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-25T16:01:05.928" pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1881gitlab # [ 390.166302] gitaly[914]: time="2026-09-25T16:01:06.034Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=116 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7548 command.oublock=0 command.real_time_ms=100 command.spawn_token_fork_ms=0 command.system_time_ms=95 command.user_time_ms=21 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPJD8JSSAJ9FEB6R8ZMN3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-25T16:01:35.228" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-25T16:01:05.928" grpc.time_ms=105.15 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1882gitlab # [ 390.280591] gitaly[914]: time="2026-09-25T16:01:06.171Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3CMPJD8JSSAJ9FEB6R8ZMN3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-25T16:01:36.069" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:06.170" grpc.time_ms=0.953 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1883gitlab # [ 390.330506] gitaly[914]: time="2026-09-25T16:01:06.221Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPJD8JSSAJ9FEB6R8ZMN3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-25T16:01:36.119" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:06.219" grpc.time_ms=0.933 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1884gitlab # [ 390.355212] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:06.245Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"00f61b4985cb1bc95e5766a3","created_at":"2026-09-25T15:59:45.647Z","trace_propagation_headers":{"sentry-trace":"548c50fbf76b48c3a77abf32d111e9bd-2d24597bc98e41d4","baggage":"sentry-trace_id=548c50fbf76b48c3a77abf32d111e9bd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3CMPJD8JSSAJ9FEB6R8ZMN3","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-25T15:59:45.653Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-00f61b4985cb1bc95e5766a3: done: 0.631261 sec","job_status":"done","queue_duration_s":79.96014,"scheduling_latency_s":79.960164,"gitaly_calls":3,"gitaly_duration_s":0.11537,"redis_calls":117,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.026704,"redis_read_bytes":455,"redis_write_bytes":28002,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001648,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001508,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.009573,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":25,"redis_queues_duration_s":0.003547,"redis_queues_read_bytes":34,"redis_queues_write_bytes":14496,"redis_queues_metadata_calls":21,"redis_queues_metadata_duration_s":0.00539,"redis_queues_metadata_read_bytes":32,"redis_queues_metadata_write_bytes":3370,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000793,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000482,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002115,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.061,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.029,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.033,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.03,"db_ci_duration_s":0.031,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.437822,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.43219728,"duration_s":0.631261,"completed_at":"2026-09-25T16:01:06.245Z","load_balancing_strategy":"primary","db_duration_s":0.072977,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1885gitlab # [ 390.423440] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:06.247Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","5a8336d76172e1ecc548fc24aea01729df24f8a5 53d1e523bb688b08820df8ef6496a15b07ee0ccf refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"3721124f4240d3717e685b9a","created_at":"2026-09-25T16:01:02.479Z","trace_propagation_headers":{"sentry-trace":"6651615a0dd94a6783c3368585113dde-ad05bb4f64f243e8","baggage":"sentry-trace_id=6651615a0dd94a6783c3368585113dde,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3CMRXBM0SVV3D8GDBMBN65N","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:02.484Z","job_size_bytes":185,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-3721124f4240d3717e685b9a: start","job_status":"start","queue_duration_s":3.762782,"scheduling_latency_s":3.762812}1886gitlab # [ 390.441311] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:06.263Z","message":"changes: \"5a8336d76172e1ecc548fc24aea01729df24f8a5 53d1e523bb688b08820df8ef6496a15b07ee0ccf refs/heads/master\\n\""}1887gitlab # [ 390.443854] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:06.312Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"6f23c1a68dec4c21242ca34f","created_at":"2026-09-25T16:01:06.310Z","trace_propagation_headers":{"sentry-trace":"6651615a0dd94a6783c3368585113dde-9521ce2410954fc7","baggage":"sentry-trace_id=6651615a0dd94a6783c3368585113dde,sentry-release=Unknown"},"correlation_id":"01M3CMRXBM0SVV3D8GDBMBN65N","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"c66b3d0d132e5a132c8a020c","job_size_bytes":46,"pid":3274,"job_status":"deduplicated","message":"ProjectCacheWorker JID-6f23c1a68dec4c21242ca34f: deduplicated: until executing","deduplication.type":"until executing"}1888gitlab # [ 390.460845] gitaly[914]: time="2026-09-25T16:01:06.293Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=576 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3CMRXBM0SVV3D8GDBMBN65N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-25T16:01:16.003" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-25T16:01:06.283" grpc.time_ms=9.89 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1889gitlab # [ 390.474438] gitaly[914]: time="2026-09-25T16:01:06.308Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMRXBM0SVV3D8GDBMBN65N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-25T16:01:16.018" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-25T16:01:06.298" grpc.time_ms=9.269 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1890gitlab # [ 390.488315] gitaly[914]: time="2026-09-25T16:01:06.335Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3CMRXBM0SVV3D8GDBMBN65N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-25T16:01:16.045" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-25T16:01:06.325" grpc.time_ms=9.918 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1891gitlab # [ 390.502657] gitaly[914]: time="2026-09-25T16:01:06.351Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=597 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3CMRXBM0SVV3D8GDBMBN65N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-09-25T16:01:36.239" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:06.339" grpc.time_ms=11.325 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1892gitlab # [ 390.518952] gitaly[914]: time="2026-09-25T16:01:06.368Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=319 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3CMRXBM0SVV3D8GDBMBN65N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-25T16:01:16.081" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-09-25T16:01:06.362" grpc.time_ms=6.321 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1893gitlab # [ 390.533026] gitaly[914]: time="2026-09-25T16:01:06.390Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M3CMRXBM0SVV3D8GDBMBN65N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-25T16:01:16.100" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-25T16:01:06.381" grpc.time_ms=9.28 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1894gitlab # [ 390.595526] gitaly[914]: time="2026-09-25T16:01:06.486Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3CMRXBM0SVV3D8GDBMBN65N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-25T16:01:16.204" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:06.484" grpc.time_ms=1.066 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1895gitlab # [ 390.616877] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:06.506Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","5a8336d76172e1ecc548fc24aea01729df24f8a5 53d1e523bb688b08820df8ef6496a15b07ee0ccf refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"3721124f4240d3717e685b9a","created_at":"2026-09-25T16:01:02.479Z","trace_propagation_headers":{"sentry-trace":"6651615a0dd94a6783c3368585113dde-ad05bb4f64f243e8","baggage":"sentry-trace_id=6651615a0dd94a6783c3368585113dde,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3CMRXBM0SVV3D8GDBMBN65N","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:02.484Z","job_size_bytes":185,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-3721124f4240d3717e685b9a: done: 0.259383 sec","job_status":"done","queue_duration_s":3.762782,"scheduling_latency_s":3.762812,"gitaly_calls":7,"gitaly_duration_s":0.073602,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.020424,"redis_read_bytes":501,"redis_write_bytes":11941,"redis_cache_calls":9,"redis_cache_duration_s":0.002292,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.001028,"redis_queues_read_bytes":9,"redis_queues_write_bytes":5718,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.003216,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.006772,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.007116,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.024,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.024,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.158623,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00075358,"exclusive_lock_hold_duration_s":0.02083202,"duration_s":0.259383,"completed_at":"2026-09-25T16:01:06.506Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0265,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1896gitlab # [ 390.681158] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:06.507Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"c66b3d0d132e5a132c8a020c","created_at":"2026-09-25T16:01:04.449Z","trace_propagation_headers":{"sentry-trace":"1889fbea594747379be5acfaf38873cf-1a4b7c9f21774249","baggage":"sentry-trace_id=1889fbea594747379be5acfaf38873cf,sentry-release=Unknown"},"correlation_id":"01M3CMPE07CPY3RA4RTWHTBTE3","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:04.471Z","job_size_bytes":46,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-c66b3d0d132e5a132c8a020c: start","job_status":"start","queue_duration_s":2.03655,"scheduling_latency_s":2.036569}1897gitlab # [ 390.698671] gitaly[914]: time="2026-09-25T16:01:06.557Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPE07CPY3RA4RTWHTBTE3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-09-25T22:01:06.551" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-25T16:01:06.552" grpc.time_ms=5.199 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1898gitlab # [ 390.709934] gitaly[914]: time="2026-09-25T16:01:06.574Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=595 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPE07CPY3RA4RTWHTBTE3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-25T16:01:36.464" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:06.564" grpc.time_ms=9.448 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1899gitlab # [ 390.723441] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:06.602Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"c66b3d0d132e5a132c8a020c","created_at":"2026-09-25T16:01:04.449Z","trace_propagation_headers":{"sentry-trace":"1889fbea594747379be5acfaf38873cf-1a4b7c9f21774249","baggage":"sentry-trace_id=1889fbea594747379be5acfaf38873cf,sentry-release=Unknown"},"correlation_id":"01M3CMPE07CPY3RA4RTWHTBTE3","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:04.471Z","job_size_bytes":46,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-c66b3d0d132e5a132c8a020c: done: 0.094442 sec","job_status":"done","queue_duration_s":2.03655,"scheduling_latency_s":2.036569,"gitaly_calls":2,"gitaly_duration_s":0.018657,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.006000999999999999,"redis_read_bytes":69,"redis_write_bytes":4886,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001152,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000499,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2114,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001323,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.002417,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":753,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00061,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.059664,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00028572,"duration_s":0.094442,"completed_at":"2026-09-25T16:01:06.602Z","load_balancing_strategy":"primary","db_duration_s":0.010212,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1900gitlab # [ 390.775381] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:06.603Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"9ea235b9cbc83cb735e8fabe","created_at":"2026-09-25T16:01:04.490Z","trace_propagation_headers":{"sentry-trace":"1889fbea594747379be5acfaf38873cf-1a4b7c9f21774249","baggage":"sentry-trace_id=1889fbea594747379be5acfaf38873cf,sentry-release=Unknown"},"correlation_id":"01M3CMPE07CPY3RA4RTWHTBTE3","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:04.496Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-9ea235b9cbc83cb735e8fabe: start","job_status":"start","queue_duration_s":2.106876,"scheduling_latency_s":2.106894}1901gitlab # [ 390.793067] gitaly[914]: time="2026-09-25T16:01:06.647Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPE07CPY3RA4RTWHTBTE3 diskcache=628e8e53-4c29-4380-a3d2-4c2c180f8434 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-25T22:01:06.626" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:06.626" pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1902gitlab # [ 390.803833] gitaly[914]: time="2026-09-25T16:01:06.647Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1175 command.oublock=0 command.real_time_ms=24 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPE07CPY3RA4RTWHTBTE3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-25T22:01:06.626" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:06.626" grpc.time_ms=20.449 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1903gitlab # [ 390.819889] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:06.687Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"9ea235b9cbc83cb735e8fabe","created_at":"2026-09-25T16:01:04.490Z","trace_propagation_headers":{"sentry-trace":"1889fbea594747379be5acfaf38873cf-1a4b7c9f21774249","baggage":"sentry-trace_id=1889fbea594747379be5acfaf38873cf,sentry-release=Unknown"},"correlation_id":"01M3CMPE07CPY3RA4RTWHTBTE3","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:04.496Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-9ea235b9cbc83cb735e8fabe: done: 0.083781 sec","job_status":"done","queue_duration_s":2.106876,"scheduling_latency_s":2.106894,"gitaly_calls":1,"gitaly_duration_s":0.022645,"redis_calls":15,"redis_duration_s":0.003581,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000337,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001138,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000872,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001234,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.01,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.018,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04045,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.05750856,"duration_s":0.083781,"completed_at":"2026-09-25T16:01:06.687Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013846,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1904gitlab # [ 390.868648] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:06.689Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"830259694a4627295513bb23","created_at":"2026-09-25T16:01:04.604Z","trace_propagation_headers":{"sentry-trace":"1889fbea594747379be5acfaf38873cf-1a4b7c9f21774249","baggage":"sentry-trace_id=1889fbea594747379be5acfaf38873cf,sentry-release=Unknown"},"correlation_id":"01M3CMPE07CPY3RA4RTWHTBTE3","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:04.606Z","job_size_bytes":23,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-830259694a4627295513bb23: start","job_status":"start","queue_duration_s":2.082933,"scheduling_latency_s":2.082956}1905gitlab # [ 390.884496] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:06.745Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"830259694a4627295513bb23","created_at":"2026-09-25T16:01:04.604Z","trace_propagation_headers":{"sentry-trace":"1889fbea594747379be5acfaf38873cf-1a4b7c9f21774249","baggage":"sentry-trace_id=1889fbea594747379be5acfaf38873cf,sentry-release=Unknown"},"correlation_id":"01M3CMPE07CPY3RA4RTWHTBTE3","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:04.606Z","job_size_bytes":23,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-830259694a4627295513bb23: done: 0.055928 sec","job_status":"done","queue_duration_s":2.082933,"scheduling_latency_s":2.082956,"gitaly_calls":1,"gitaly_duration_s":0.011483,"redis_calls":16,"redis_duration_s":0.003467,"redis_read_bytes":52,"redis_write_bytes":2049,"redis_queues_calls":1,"redis_queues_duration_s":0.000294,"redis_queues_read_bytes":1,"redis_queues_write_bytes":980,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001027,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.001489,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000657,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03666,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.055928,"completed_at":"2026-09-25T16:01:06.745Z","load_balancing_strategy":"primary","db_duration_s":0.002964,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1906gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=53d1e523bb688b08820df8ef6496a15b07ee0ccf'1907gitlab # [ 390.930908] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:06.747Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"c2dc2a10e7f5f62e971e756b","created_at":"2026-09-25T16:01:05.210Z","trace_propagation_headers":{"sentry-trace":"1889fbea594747379be5acfaf38873cf-1a4b7c9f21774249","baggage":"sentry-trace_id=1889fbea594747379be5acfaf38873cf,sentry-release=Unknown"},"correlation_id":"01M3CMPE07CPY3RA4RTWHTBTE3","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6208a7521a78bd782559e603312acae70893162cb632067d5b4537f71d541b35","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:05.216Z","job_size_bytes":467,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-c2dc2a10e7f5f62e971e756b: start","job_status":"start","queue_duration_s":1.530991,"scheduling_latency_s":1.531017}1908gitlab # [ 390.948395] gitaly[914]: time="2026-09-25T16:01:06.733Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=596 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPE07CPY3RA4RTWHTBTE3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-25T16:01:36.624" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:06.724" grpc.time_ms=9.203 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1909gitlab # [ 390.961613] gitaly[914]: time="2026-09-25T16:01:06.781Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPE07CPY3RA4RTWHTBTE3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-25T16:01:36.679" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:06.780" grpc.time_ms=0.651 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1910gitlab # [ 390.974258] gitaly[914]: time="2026-09-25T16:01:06.792Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=3 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMPE07CPY3RA4RTWHTBTE3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-25T16:01:16.013" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-25T16:01:06.783" grpc.time_ms=8.738 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1911gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=53d1e523bb688b08820df8ef6496a15b07ee0ccf', in 0.04 seconds)1912gitlab # [ 390.988137] nixbot[3401]: INFO: ::ffff:127.0.0.1:59740 - "GET /api/repos/gitlab/root/test-flake/builds?commit=53d1e523bb688b08820df8ef6496a15b07ee0ccf HTTP/1.1" 200 OK1913gitlab # [ 391.056438] gitaly[914]: time="2026-09-25T16:01:06.947Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3CMPE07CPY3RA4RTWHTBTE3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-25T16:01:36.845" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:06.946" grpc.time_ms=0.834 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1914gitlab # [ 391.078028] gitaly[914]: time="2026-09-25T16:01:06.953Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=49 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M3CMPE07CPY3RA4RTWHTBTE3 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-25T16:01:36.850" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:06.951" grpc.time_ms=2.575 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1915gitlab # [ 391.157707] gitaly[914]: time="2026-09-25T16:01:07.048Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3CMPE07CPY3RA4RTWHTBTE3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-25T16:01:17.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-25T16:01:07.038" grpc.time_ms=9.201 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1916gitlab # [ 391.704158] gitaly[914]: time="2026-09-25T16:01:07.594Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=880 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M3CMPE07CPY3RA4RTWHTBTE3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-25T16:01:37.478" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:07.578" grpc.time_ms=15.892 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1917gitlab # [ 391.722510] gitaly[914]: time="2026-09-25T16:01:07.613Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M3CMPE07CPY3RA4RTWHTBTE3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-25T16:01:37.498" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:07.598" grpc.time_ms=14.402 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1918gitlab # [ 391.759467] sidekiq-cluster[3274]: {"severity":"WARN","time":"2026-09-25T16:01:07.649Z","class":"CreatePipelineWorker","correlation_id":"01M3CMPE07CPY3RA4RTWHTBTE3","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"5a8336d76172e1ecc548fc24aea01729df24f8a5","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"5a8336d76172e1ecc548fc24aea01729df24f8a5","ref":"refs/heads/master"},"retry":0}1919gitlab # [ 391.769002] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:07.659Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"c2dc2a10e7f5f62e971e756b","created_at":"2026-09-25T16:01:05.210Z","trace_propagation_headers":{"sentry-trace":"1889fbea594747379be5acfaf38873cf-1a4b7c9f21774249","baggage":"sentry-trace_id=1889fbea594747379be5acfaf38873cf,sentry-release=Unknown"},"correlation_id":"01M3CMPE07CPY3RA4RTWHTBTE3","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6208a7521a78bd782559e603312acae70893162cb632067d5b4537f71d541b35","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:05.216Z","job_size_bytes":467,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-c2dc2a10e7f5f62e971e756b: done: 0.911267 sec","job_status":"done","queue_duration_s":1.530991,"scheduling_latency_s":1.531017,"gitaly_calls":7,"gitaly_duration_s":0.0685,"redis_calls":51,"redis_duration_s":0.00917,"redis_read_bytes":169,"redis_write_bytes":4168,"redis_cache_calls":4,"redis_cache_duration_s":0.000888,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.003727,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001186,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002346,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001023,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.79551,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003284,"duration_s":0.911267,"completed_at":"2026-09-25T16:01:07.658Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.051217,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1920gitlab # [ 391.821870] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:07.661Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"0c1231f1-218e-49f8-a78c-d11e877e6803\"}"],"class":"WebHookWorker","jid":"5f36a1e1b0a71faf0174bb12","created_at":"2026-09-25T16:01:05.363Z","trace_propagation_headers":{"sentry-trace":"1889fbea594747379be5acfaf38873cf-1a4b7c9f21774249","baggage":"sentry-trace_id=1889fbea594747379be5acfaf38873cf,sentry-release=Unknown"},"correlation_id":"01M3CMPE07CPY3RA4RTWHTBTE3","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4055d366afb6b714226e017c97336e17caa3ae309bcd6c9c020ea18bb4202be4","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:05.368Z","job_size_bytes":1751,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-5f36a1e1b0a71faf0174bb12: start","job_status":"start","queue_duration_s":2.292586,"scheduling_latency_s":2.292612}1921gitlab # [ 391.930766] nixbot[3401]: INFO: ::ffff:127.0.0.1:59756 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted1922gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=53d1e523bb688b08820df8ef6496a15b07ee0ccf'1923gitlab # [ 391.955604] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:07.845Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"0c1231f1-218e-49f8-a78c-d11e877e6803\"}"],"class":"WebHookWorker","jid":"5f36a1e1b0a71faf0174bb12","created_at":"2026-09-25T16:01:05.363Z","trace_propagation_headers":{"sentry-trace":"1889fbea594747379be5acfaf38873cf-1a4b7c9f21774249","baggage":"sentry-trace_id=1889fbea594747379be5acfaf38873cf,sentry-release=Unknown"},"correlation_id":"01M3CMPE07CPY3RA4RTWHTBTE3","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4055d366afb6b714226e017c97336e17caa3ae309bcd6c9c020ea18bb4202be4","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:05.368Z","job_size_bytes":1751,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-5f36a1e1b0a71faf0174bb12: done: 0.184159 sec","job_status":"done","queue_duration_s":2.292586,"scheduling_latency_s":2.292612,"redis_calls":19,"redis_duration_s":0.004402,"redis_read_bytes":32,"redis_write_bytes":5368,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00114,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000496,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3730,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001894,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000872,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.06635642000003372,"cpu_s":0.102783,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00055288,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.184159,"completed_at":"2026-09-25T16:01:07.845Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003271,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1924gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=53d1e523bb688b08820df8ef6496a15b07ee0ccf', in 0.04 seconds)1925gitlab # [ 392.028924] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:07.848Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"4a474c236de4a582b9e19f55","created_at":"2026-09-25T16:01:05.905Z","trace_propagation_headers":{"sentry-trace":"548c50fbf76b48c3a77abf32d111e9bd-1e1f64d608fe4e0d","baggage":"sentry-trace_id=548c50fbf76b48c3a77abf32d111e9bd,sentry-release=Unknown"},"correlation_id":"01M3CMPJD8JSSAJ9FEB6R8ZMN3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:05.912Z","job_size_bytes":23,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-4a474c236de4a582b9e19f55: start","job_status":"start","queue_duration_s":1.936416,"scheduling_latency_s":1.936446}1926gitlab # [ 392.050847] nixbot[3401]: INFO: ::ffff:127.0.0.1:59772 - "GET /api/repos/gitlab/root/test-flake/builds?commit=53d1e523bb688b08820df8ef6496a15b07ee0ccf HTTP/1.1" 200 OK1927gitlab # [ 392.079435] gitaly[914]: time="2026-09-25T16:01:07.969Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3CMS38RKCFQPJN8X1YKEHWD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T16:01:07.958" grpc.time_ms=10.474 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc1928gitlab # [ 392.098088] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:16:01:07 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1051929gitlab # [ 392.109714] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:07.999Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"4a474c236de4a582b9e19f55","created_at":"2026-09-25T16:01:05.905Z","trace_propagation_headers":{"sentry-trace":"548c50fbf76b48c3a77abf32d111e9bd-1e1f64d608fe4e0d","baggage":"sentry-trace_id=548c50fbf76b48c3a77abf32d111e9bd,sentry-release=Unknown"},"correlation_id":"01M3CMPJD8JSSAJ9FEB6R8ZMN3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:05.912Z","job_size_bytes":23,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-4a474c236de4a582b9e19f55: done: 0.150968 sec","job_status":"done","queue_duration_s":1.936416,"scheduling_latency_s":1.936446,"redis_calls":10,"redis_duration_s":0.00297,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000846,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001566,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000558,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.093523,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00030332,"duration_s":0.150968,"completed_at":"2026-09-25T16:01:07.999Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.034385,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1930gitlab # [ 392.165684] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:08.001Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"0987bc183ba1b4b2c16bba03","created_at":"2026-09-25T16:01:06.039Z","trace_propagation_headers":{"sentry-trace":"548c50fbf76b48c3a77abf32d111e9bd-1e1f64d608fe4e0d","baggage":"sentry-trace_id=548c50fbf76b48c3a77abf32d111e9bd,sentry-release=Unknown"},"correlation_id":"01M3CMPJD8JSSAJ9FEB6R8ZMN3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:06.046Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-0987bc183ba1b4b2c16bba03: start","job_status":"start","queue_duration_s":1.954906,"scheduling_latency_s":1.954931}1931gitlab # [ 392.192282] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:08.042Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"0987bc183ba1b4b2c16bba03","created_at":"2026-09-25T16:01:06.039Z","trace_propagation_headers":{"sentry-trace":"548c50fbf76b48c3a77abf32d111e9bd-1e1f64d608fe4e0d","baggage":"sentry-trace_id=548c50fbf76b48c3a77abf32d111e9bd,sentry-release=Unknown"},"correlation_id":"01M3CMPJD8JSSAJ9FEB6R8ZMN3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:06.046Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-0987bc183ba1b4b2c16bba03: done: 0.040517 sec","job_status":"done","queue_duration_s":1.954906,"scheduling_latency_s":1.954931,"redis_calls":10,"redis_duration_s":0.003212,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000369,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00243,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000413,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026067,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00166186,"duration_s":0.040517,"completed_at":"2026-09-25T16:01:08.042Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005698,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1932gitlab # [ 392.247172] gitaly[914]: time="2026-09-25T16:01:08.048Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3CMS3C4Z09G3SN75RKDYMAX grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T16:01:08.036" pid=914 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1933gitlab # [ 392.258554] gitlab-workhorse[821]: time="2026-09-25T16:01:08Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671934gitlab # [ 392.260671] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:16:01:08 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 761935gitlab # [ 392.262716] gitlab-workhorse[821]: time="2026-09-25T16:01:08Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3371936gitlab # [ 392.264799] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:16:01:08 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 337 "" "git/2.55.0" 801937gitlab # [ 392.266897] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:08.043Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"5dbc605bd8223f21e0b0381f","created_at":"2026-09-25T16:01:06.064Z","trace_propagation_headers":{"sentry-trace":"548c50fbf76b48c3a77abf32d111e9bd-1e1f64d608fe4e0d","baggage":"sentry-trace_id=548c50fbf76b48c3a77abf32d111e9bd,sentry-release=Unknown"},"correlation_id":"01M3CMPJD8JSSAJ9FEB6R8ZMN3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:06.071Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-5dbc605bd8223f21e0b0381f: start","job_status":"start","queue_duration_s":1.971987,"scheduling_latency_s":1.972004}1938gitlab # [ 392.283952] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:08.082Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"5dbc605bd8223f21e0b0381f","created_at":"2026-09-25T16:01:06.064Z","trace_propagation_headers":{"sentry-trace":"548c50fbf76b48c3a77abf32d111e9bd-1e1f64d608fe4e0d","baggage":"sentry-trace_id=548c50fbf76b48c3a77abf32d111e9bd,sentry-release=Unknown"},"correlation_id":"01M3CMPJD8JSSAJ9FEB6R8ZMN3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:06.071Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-5dbc605bd8223f21e0b0381f: done: 0.038808 sec","job_status":"done","queue_duration_s":1.971987,"scheduling_latency_s":1.972004,"redis_calls":10,"redis_duration_s":0.00139,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000297,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000804,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000289,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025306,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00023202,"duration_s":0.038808,"completed_at":"2026-09-25T16:01:08.082Z","load_balancing_strategy":"primary","db_duration_s":0.006737,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1939gitlab # [ 392.330180] gitaly[914]: time="2026-09-25T16:01:08.048Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMS3C4Z09G3SN75RKDYMAX grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T16:01:08.036" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc1940gitlab # [ 392.343907] nixbot[3401]: {"ts": "2026-09-25T16:01:08.165Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"}1941gitlab # [ 392.346669] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:08.084Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"de87cad140662495ac5c404a","created_at":"2026-09-25T16:01:06.089Z","trace_propagation_headers":{"sentry-trace":"548c50fbf76b48c3a77abf32d111e9bd-1e1f64d608fe4e0d","baggage":"sentry-trace_id=548c50fbf76b48c3a77abf32d111e9bd,sentry-release=Unknown"},"correlation_id":"01M3CMPJD8JSSAJ9FEB6R8ZMN3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:06.095Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-de87cad140662495ac5c404a: start","job_status":"start","queue_duration_s":1.988738,"scheduling_latency_s":1.988755}1942gitlab # [ 392.362875] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:08.127Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"de87cad140662495ac5c404a","created_at":"2026-09-25T16:01:06.089Z","trace_propagation_headers":{"sentry-trace":"548c50fbf76b48c3a77abf32d111e9bd-1e1f64d608fe4e0d","baggage":"sentry-trace_id=548c50fbf76b48c3a77abf32d111e9bd,sentry-release=Unknown"},"correlation_id":"01M3CMPJD8JSSAJ9FEB6R8ZMN3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:06.095Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-de87cad140662495ac5c404a: done: 0.043129 sec","job_status":"done","queue_duration_s":1.988738,"scheduling_latency_s":1.988755,"redis_calls":13,"redis_duration_s":0.0025369999999999998,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00048,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001259,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000415,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000383,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026975,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00052248,"duration_s":0.043129,"completed_at":"2026-09-25T16:01:08.127Z","load_balancing_strategy":"primary","db_duration_s":0.009591,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1943gitlab # [ 392.409860] gitaly[914]: time="2026-09-25T16:01:08.048Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMS3C4Z09G3SN75RKDYMAX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T16:01:08.036" grpc.time_ms=12.102 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc1944gitlab # [ 392.425323] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:08.128Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"ce62422dd6af9e3227f5d7e0","created_at":"2026-09-25T16:01:06.096Z","trace_propagation_headers":{"sentry-trace":"548c50fbf76b48c3a77abf32d111e9bd-1e1f64d608fe4e0d","baggage":"sentry-trace_id=548c50fbf76b48c3a77abf32d111e9bd,sentry-release=Unknown"},"correlation_id":"01M3CMPJD8JSSAJ9FEB6R8ZMN3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:06.102Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-ce62422dd6af9e3227f5d7e0: start","job_status":"start","queue_duration_s":2.026486,"scheduling_latency_s":2.026502}1945gitlab # [ 392.443047] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:08.183Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"ce62422dd6af9e3227f5d7e0","created_at":"2026-09-25T16:01:06.096Z","trace_propagation_headers":{"sentry-trace":"548c50fbf76b48c3a77abf32d111e9bd-1e1f64d608fe4e0d","baggage":"sentry-trace_id=548c50fbf76b48c3a77abf32d111e9bd,sentry-release=Unknown"},"correlation_id":"01M3CMPJD8JSSAJ9FEB6R8ZMN3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:06.102Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-ce62422dd6af9e3227f5d7e0: done: 0.054823 sec","job_status":"done","queue_duration_s":2.026486,"scheduling_latency_s":2.026502,"redis_calls":10,"redis_duration_s":0.003418,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001886,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001179,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000353,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033473,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00031508,"duration_s":0.054823,"completed_at":"2026-09-25T16:01:08.183Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009276,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1946gitlab # [ 392.489951] gitaly[914]: time="2026-09-25T16:01:08.130Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3CMS3EK5DEH0N86DJ5XF24N grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T16:01:08.110" pid=914 remote_ip="2001:db8:1::1" request_sha=8e98cb6a2f62f8535cb016a636f3e6ac8d8ad44c response_bytes=337 span.kind=server system=grpc1947gitlab # [ 392.500492] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:08.185Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"c206f820779236603f54de50","created_at":"2026-09-25T16:01:06.102Z","trace_propagation_headers":{"sentry-trace":"548c50fbf76b48c3a77abf32d111e9bd-1e1f64d608fe4e0d","baggage":"sentry-trace_id=548c50fbf76b48c3a77abf32d111e9bd,sentry-release=Unknown"},"correlation_id":"01M3CMPJD8JSSAJ9FEB6R8ZMN3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:06.109Z","job_size_bytes":19,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-c206f820779236603f54de50: start","job_status":"start","queue_duration_s":2.075315,"scheduling_latency_s":2.075332}1948gitlab # [ 392.516987] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:08.223Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"c206f820779236603f54de50","created_at":"2026-09-25T16:01:06.102Z","trace_propagation_headers":{"sentry-trace":"548c50fbf76b48c3a77abf32d111e9bd-1e1f64d608fe4e0d","baggage":"sentry-trace_id=548c50fbf76b48c3a77abf32d111e9bd,sentry-release=Unknown"},"correlation_id":"01M3CMPJD8JSSAJ9FEB6R8ZMN3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:06.109Z","job_size_bytes":19,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-c206f820779236603f54de50: done: 0.03823 sec","job_status":"done","queue_duration_s":2.075315,"scheduling_latency_s":2.075332,"redis_calls":11,"redis_duration_s":0.004643,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00036,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003755,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000528,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020968,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00238166,"duration_s":0.03823,"completed_at":"2026-09-25T16:01:08.223Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008074,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1949gitlab # [ 392.561926] gitaly[914]: time="2026-09-25T16:01:08.130Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1129 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=0 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMS3EK5DEH0N86DJ5XF24N grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T16:01:08.110" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc1950gitlab # [ 392.576125] nixbot[3401]: {"ts": "2026-09-25T16:01:08.395Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/5a8336d76172e1ecc548fc24aea01729df24f8a5 \"HTTP/1.1 201 Created\""}1951gitlab # [ 392.578933] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:08.224Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"098fe56482500064cef454d9","created_at":"2026-09-25T16:01:06.113Z","trace_propagation_headers":{"sentry-trace":"548c50fbf76b48c3a77abf32d111e9bd-1e1f64d608fe4e0d","baggage":"sentry-trace_id=548c50fbf76b48c3a77abf32d111e9bd,sentry-release=Unknown"},"correlation_id":"01M3CMPJD8JSSAJ9FEB6R8ZMN3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:06.119Z","job_size_bytes":66,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-098fe56482500064cef454d9: start","job_status":"start","queue_duration_s":2.10535,"scheduling_latency_s":2.105366}1952gitlab # [ 392.596790] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:08.267Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"098fe56482500064cef454d9","created_at":"2026-09-25T16:01:06.113Z","trace_propagation_headers":{"sentry-trace":"548c50fbf76b48c3a77abf32d111e9bd-1e1f64d608fe4e0d","baggage":"sentry-trace_id=548c50fbf76b48c3a77abf32d111e9bd,sentry-release=Unknown"},"correlation_id":"01M3CMPJD8JSSAJ9FEB6R8ZMN3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:06.119Z","job_size_bytes":66,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-098fe56482500064cef454d9: done: 0.043219 sec","job_status":"done","queue_duration_s":2.10535,"scheduling_latency_s":2.105366,"redis_calls":10,"redis_duration_s":0.0029379999999999996,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000553,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002005,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00038,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02566,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00035324,"duration_s":0.043219,"completed_at":"2026-09-25T16:01:08.267Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008801,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1953gitlab # [ 392.643234] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:16:01:08 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/5a8336d76172e1ecc548fc24aea01729df24f8a5 HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 2161954gitlab # [ 392.645919] gitaly[914]: time="2026-09-25T16:01:08.131Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1129 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=0 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMS3EK5DEH0N86DJ5XF24N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T16:01:08.110" grpc.time_ms=21.246 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc1955gitlab # [ 392.661204] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:08.269Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"8ad651f83e6da61a5e4d53d1","created_at":"2026-09-25T16:01:06.120Z","trace_propagation_headers":{"sentry-trace":"548c50fbf76b48c3a77abf32d111e9bd-1e1f64d608fe4e0d","baggage":"sentry-trace_id=548c50fbf76b48c3a77abf32d111e9bd,sentry-release=Unknown"},"correlation_id":"01M3CMPJD8JSSAJ9FEB6R8ZMN3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:06.125Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-8ad651f83e6da61a5e4d53d1: start","job_status":"start","queue_duration_s":2.144409,"scheduling_latency_s":2.144438}1956gitlab # [ 392.677667] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:08.325Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"8ad651f83e6da61a5e4d53d1","created_at":"2026-09-25T16:01:06.120Z","trace_propagation_headers":{"sentry-trace":"548c50fbf76b48c3a77abf32d111e9bd-1e1f64d608fe4e0d","baggage":"sentry-trace_id=548c50fbf76b48c3a77abf32d111e9bd,sentry-release=Unknown"},"correlation_id":"01M3CMPJD8JSSAJ9FEB6R8ZMN3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:06.125Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-8ad651f83e6da61a5e4d53d1: done: 0.056123 sec","job_status":"done","queue_duration_s":2.144409,"scheduling_latency_s":2.144438,"redis_calls":10,"redis_duration_s":0.002304,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000355,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001296,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000653,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041553,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00041238,"duration_s":0.056123,"completed_at":"2026-09-25T16:01:08.325Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006163,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1957gitlab # [ 392.724071] nixbot[3401]: {"ts": "2026-09-25T16:01:08.549Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/5a8336d76172e1ecc548fc24aea01729df24f8a5 \"HTTP/1.1 201 Created\""}1958gitlab # [ 392.726883] gitaly[914]: time="2026-09-25T16:01:08.262Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMS3JH6R05MZ1PDB193R9V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-25T16:01:38.161" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:08.261" grpc.time_ms=0.688 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1959gitlab # [ 392.740099] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:16:01:08 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/5a8336d76172e1ecc548fc24aea01729df24f8a5 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1471960gitlab # [ 392.742688] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:08.327Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"2b88e6aa9b07d60fccb6eed9","created_at":"2026-09-25T16:01:06.132Z","trace_propagation_headers":{"sentry-trace":"548c50fbf76b48c3a77abf32d111e9bd-1e1f64d608fe4e0d","baggage":"sentry-trace_id=548c50fbf76b48c3a77abf32d111e9bd,sentry-release=Unknown"},"correlation_id":"01M3CMPJD8JSSAJ9FEB6R8ZMN3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:06.137Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-2b88e6aa9b07d60fccb6eed9: start","job_status":"start","queue_duration_s":2.190383,"scheduling_latency_s":2.190406}1961gitlab # [ 392.759982] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:08.372Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"2b88e6aa9b07d60fccb6eed9","created_at":"2026-09-25T16:01:06.132Z","trace_propagation_headers":{"sentry-trace":"548c50fbf76b48c3a77abf32d111e9bd-1e1f64d608fe4e0d","baggage":"sentry-trace_id=548c50fbf76b48c3a77abf32d111e9bd,sentry-release=Unknown"},"correlation_id":"01M3CMPJD8JSSAJ9FEB6R8ZMN3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:06.137Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-2b88e6aa9b07d60fccb6eed9: done: 0.044582 sec","job_status":"done","queue_duration_s":2.190383,"scheduling_latency_s":2.190406,"redis_calls":10,"redis_duration_s":0.002499,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000359,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001483,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000657,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027187,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00024834,"duration_s":0.044582,"completed_at":"2026-09-25T16:01:08.372Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009738,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1962gitlab # [ 392.806258] gitaly[914]: time="2026-09-25T16:01:08.335Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3CMS3JH6R05MZ1PDB193R9V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-25T16:01:18.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-25T16:01:08.327" grpc.time_ms=8.019 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1963gitlab # [ 392.820073] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:08.373Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"c45e4d0e2db40691ba5f4b07","created_at":"2026-09-25T16:01:06.137Z","trace_propagation_headers":{"sentry-trace":"548c50fbf76b48c3a77abf32d111e9bd-1e1f64d608fe4e0d","baggage":"sentry-trace_id=548c50fbf76b48c3a77abf32d111e9bd,sentry-release=Unknown"},"correlation_id":"01M3CMPJD8JSSAJ9FEB6R8ZMN3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:06.142Z","job_size_bytes":6,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-c45e4d0e2db40691ba5f4b07: start","job_status":"start","queue_duration_s":2.230965,"scheduling_latency_s":2.230985}1964gitlab # [ 392.836430] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:08.399Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"c45e4d0e2db40691ba5f4b07","created_at":"2026-09-25T16:01:06.137Z","trace_propagation_headers":{"sentry-trace":"548c50fbf76b48c3a77abf32d111e9bd-1e1f64d608fe4e0d","baggage":"sentry-trace_id=548c50fbf76b48c3a77abf32d111e9bd,sentry-release=Unknown"},"correlation_id":"01M3CMPJD8JSSAJ9FEB6R8ZMN3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:06.142Z","job_size_bytes":6,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-c45e4d0e2db40691ba5f4b07: done: 0.025833 sec","job_status":"done","queue_duration_s":2.230965,"scheduling_latency_s":2.230985,"redis_calls":10,"redis_duration_s":0.001852,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000494,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000901,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000457,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016408,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00044728,"duration_s":0.025833,"completed_at":"2026-09-25T16:01:08.399Z","load_balancing_strategy":"primary","db_duration_s":0.002549,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1965gitlab # [ 392.881231] gitaly[914]: time="2026-09-25T16:01:08.457Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMS3SGDBFYM8MM6RCA63X4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-25T16:01:38.355" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:08.456" grpc.time_ms=0.603 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1966gitlab # [ 392.895177] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:08.400Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"31c1a286c1fa1c8ab11261d2","created_at":"2026-09-25T16:01:06.143Z","trace_propagation_headers":{"sentry-trace":"548c50fbf76b48c3a77abf32d111e9bd-1e1f64d608fe4e0d","baggage":"sentry-trace_id=548c50fbf76b48c3a77abf32d111e9bd,sentry-release=Unknown"},"correlation_id":"01M3CMPJD8JSSAJ9FEB6R8ZMN3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:06.147Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-31c1a286c1fa1c8ab11261d2: start","job_status":"start","queue_duration_s":2.253512,"scheduling_latency_s":2.253528}1967gitlab # [ 392.910978] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:08.442Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"31c1a286c1fa1c8ab11261d2","created_at":"2026-09-25T16:01:06.143Z","trace_propagation_headers":{"sentry-trace":"548c50fbf76b48c3a77abf32d111e9bd-1e1f64d608fe4e0d","baggage":"sentry-trace_id=548c50fbf76b48c3a77abf32d111e9bd,sentry-release=Unknown"},"correlation_id":"01M3CMPJD8JSSAJ9FEB6R8ZMN3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:06.147Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-31c1a286c1fa1c8ab11261d2: done: 0.041745 sec","job_status":"done","queue_duration_s":2.253512,"scheduling_latency_s":2.253528,"redis_calls":10,"redis_duration_s":0.002035,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00037,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001242,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000423,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027999,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026382,"duration_s":0.041745,"completed_at":"2026-09-25T16:01:08.442Z","load_balancing_strategy":"primary","db_duration_s":0.007679,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1968gitlab # [ 392.955460] gitaly[914]: time="2026-09-25T16:01:08.499Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M3CMS3SGDBFYM8MM6RCA63X4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-25T16:01:18.171" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-25T16:01:08.492" grpc.time_ms=7.146 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1969gitlab # [ 392.969398] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:08.444Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"da1dd3a9d911b77cd2932f33","created_at":"2026-09-25T16:01:06.153Z","trace_propagation_headers":{"sentry-trace":"548c50fbf76b48c3a77abf32d111e9bd-1e1f64d608fe4e0d","baggage":"sentry-trace_id=548c50fbf76b48c3a77abf32d111e9bd,sentry-release=Unknown"},"correlation_id":"01M3CMPJD8JSSAJ9FEB6R8ZMN3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:06.157Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-da1dd3a9d911b77cd2932f33: start","job_status":"start","queue_duration_s":2.286376,"scheduling_latency_s":2.286394}1970gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=53d1e523bb688b08820df8ef6496a15b07ee0ccf'1971gitlab # [ 392.986023] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:08.641Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"da1dd3a9d911b77cd2932f33","created_at":"2026-09-25T16:01:06.153Z","trace_propagation_headers":{"sentry-trace":"548c50fbf76b48c3a77abf32d111e9bd-1e1f64d608fe4e0d","baggage":"sentry-trace_id=548c50fbf76b48c3a77abf32d111e9bd,sentry-release=Unknown"},"correlation_id":"01M3CMPJD8JSSAJ9FEB6R8ZMN3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:06.157Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-da1dd3a9d911b77cd2932f33: done: 0.197595 sec","job_status":"done","queue_duration_s":2.286376,"scheduling_latency_s":2.286394,"redis_calls":14,"redis_duration_s":0.004959,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000896,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001827,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002236,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.044381,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0005592,"duration_s":0.197595,"completed_at":"2026-09-25T16:01:08.641Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013539,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1972gitlab # [ 393.031168] gitaly[914]: time="2026-09-25T16:01:08.641Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMS3Y984H8B5XZ1AEKYF5H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-25T16:01:38.640" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:08.640" grpc.time_ms=0.662 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1973gitlab # [ 393.043422] gitaly[914]: time="2026-09-25T16:01:08.671Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3CMRXBM0SVV3D8GDBMBN65N diskcache=b892ab04-80ce-4fb9-bd60-d177164f9b92 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-25T22:01:08.662" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:08.663" pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1974gitlab # [ 393.053904] nixbot[3401]: {"ts": "2026-09-25T16:01:08.904Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/5a8336d76172e1ecc548fc24aea01729df24f8a5 \"HTTP/1.1 201 Created\""}1975gitlab # [ 393.056832] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:08.643Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"5a7460d44f7102e674dd4dba","created_at":"2026-09-25T16:01:06.318Z","trace_propagation_headers":{"sentry-trace":"6651615a0dd94a6783c3368585113dde-9521ce2410954fc7","baggage":"sentry-trace_id=6651615a0dd94a6783c3368585113dde,sentry-release=Unknown"},"correlation_id":"01M3CMRXBM0SVV3D8GDBMBN65N","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"9ea235b9cbc83cb735e8fabe","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:06.321Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-5a7460d44f7102e674dd4dba: start","job_status":"start","queue_duration_s":2.321952,"scheduling_latency_s":2.321978}1976gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=53d1e523bb688b08820df8ef6496a15b07ee0ccf', in 0.05 seconds)1977gitlab # [ 393.074145] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:08.687Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"5a7460d44f7102e674dd4dba","created_at":"2026-09-25T16:01:06.318Z","trace_propagation_headers":{"sentry-trace":"6651615a0dd94a6783c3368585113dde-9521ce2410954fc7","baggage":"sentry-trace_id=6651615a0dd94a6783c3368585113dde,sentry-release=Unknown"},"correlation_id":"01M3CMRXBM0SVV3D8GDBMBN65N","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"9ea235b9cbc83cb735e8fabe","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:06.321Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-5a7460d44f7102e674dd4dba: done: 0.043856 sec","job_status":"done","queue_duration_s":2.321952,"scheduling_latency_s":2.321978,"gitaly_calls":1,"gitaly_duration_s":0.010546,"redis_calls":12,"redis_duration_s":0.002171,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000894,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000388,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000889,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025817,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.02514368,"duration_s":0.043856,"completed_at":"2026-09-25T16:01:08.687Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004727,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1978gitlab # [ 393.122814] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:16:01:08 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/5a8336d76172e1ecc548fc24aea01729df24f8a5 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 3481979gitlab # [ 393.125477] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:16:01:09 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 671980gitlab # [ 393.127867] gitaly[914]: time="2026-09-25T16:01:08.671Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=290 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMRXBM0SVV3D8GDBMBN65N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-25T22:01:08.662" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:08.663" grpc.time_ms=7.561 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1981gitlab # [ 393.141658] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:08.689Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"e429ae47d80ab5a7c3ecd5eb","created_at":"2026-09-25T16:01:06.447Z","trace_propagation_headers":{"sentry-trace":"6651615a0dd94a6783c3368585113dde-9521ce2410954fc7","baggage":"sentry-trace_id=6651615a0dd94a6783c3368585113dde,sentry-release=Unknown"},"correlation_id":"01M3CMRXBM0SVV3D8GDBMBN65N","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:01ff7c5e2852721e05ff5756c6d4a33b9080a1ea2814ffefeacf179855b02d6e","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:06.450Z","job_size_bytes":467,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-e429ae47d80ab5a7c3ecd5eb: start","job_status":"start","queue_duration_s":2.238829,"scheduling_latency_s":2.238851}1982gitlab # [ 393.159349] nixbot[3401]: INFO: ::ffff:127.0.0.1:59784 - "GET /api/repos/gitlab/root/test-flake/builds?commit=53d1e523bb688b08820df8ef6496a15b07ee0ccf HTTP/1.1" 200 OK1983gitlab # [ 393.161402] gitaly[914]: time="2026-09-25T16:01:08.715Z" level=info msg="finished unary call with code OK" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=2 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMRXBM0SVV3D8GDBMBN65N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-25T16:01:38.611" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:08.712" grpc.time_ms=2.541 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1984gitlab # [ 393.174102] gitaly[914]: time="2026-09-25T16:01:08.725Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMRXBM0SVV3D8GDBMBN65N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-25T16:01:18.008" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-25T16:01:08.718" grpc.time_ms=7.173 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1985gitlab # [ 393.187551] gitaly[914]: time="2026-09-25T16:01:08.762Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3CMRXBM0SVV3D8GDBMBN65N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-25T16:01:38.661" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:08.761" grpc.time_ms=0.814 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1986gitlab # [ 393.200638] gitlab-workhorse[821]: time="2026-09-25T16:01:09Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671987gitlab # [ 393.202506] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:16:01:09 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 611988gitlab # [ 393.204418] gitaly[914]: time="2026-09-25T16:01:08.768Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=41 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M3CMRXBM0SVV3D8GDBMBN65N error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-25T16:01:38.665" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:08.765" grpc.time_ms=2.642 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1989gitlab # [ 393.220898] gitaly[914]: time="2026-09-25T16:01:08.777Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M3CMS3Y984H8B5XZ1AEKYF5H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-25T16:01:18.000" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-25T16:01:08.771" grpc.time_ms=6.706 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1990gitlab # [ 393.234534] gitaly[914]: time="2026-09-25T16:01:08.805Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M3CMRXBM0SVV3D8GDBMBN65N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-25T16:01:18.088" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-25T16:01:08.798" grpc.time_ms=6.942 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1991gitlab # [ 393.248463] gitaly[914]: time="2026-09-25T16:01:09.004Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M3CMS4A9S49NQGB0YKXEQFZF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=194 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T16:01:08.995" grpc.time_ms=9.011 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc1992gitlab # [ 393.269872] gitaly[914]: time="2026-09-25T16:01:09.066Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3CMS4CEWPYGN1TEQRKS3N4V grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T16:01:09.057" pid=914 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1993gitlab # [ 393.286386] gitaly[914]: time="2026-09-25T16:01:09.066Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMS4CEWPYGN1TEQRKS3N4V grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T16:01:09.057" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc1994gitlab # [ 393.308741] gitaly[914]: time="2026-09-25T16:01:09.067Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMS4CEWPYGN1TEQRKS3N4V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T16:01:09.057" grpc.time_ms=9.356 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc1995gitlab # [ 393.431280] gitaly[914]: time="2026-09-25T16:01:09.321Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=881 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=7 component=gitaly.StreamServerInterceptor correlation_id=01M3CMRXBM0SVV3D8GDBMBN65N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-25T16:01:39.206" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:09.306" grpc.time_ms=15.347 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1996gitlab # [ 393.449049] gitaly[914]: time="2026-09-25T16:01:09.339Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M3CMRXBM0SVV3D8GDBMBN65N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-25T16:01:39.225" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:09.325" grpc.time_ms=13.941 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1997gitlab # [ 393.465575] sidekiq-cluster[3274]: {"severity":"WARN","time":"2026-09-25T16:01:09.356Z","class":"CreatePipelineWorker","correlation_id":"01M3CMRXBM0SVV3D8GDBMBN65N","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"5a8336d76172e1ecc548fc24aea01729df24f8a5","after":"53d1e523bb688b08820df8ef6496a15b07ee0ccf","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"53d1e523bb688b08820df8ef6496a15b07ee0ccf","ref":"refs/heads/master"},"retry":0}1998gitlab # [ 393.473503] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:09.363Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"e429ae47d80ab5a7c3ecd5eb","created_at":"2026-09-25T16:01:06.447Z","trace_propagation_headers":{"sentry-trace":"6651615a0dd94a6783c3368585113dde-9521ce2410954fc7","baggage":"sentry-trace_id=6651615a0dd94a6783c3368585113dde,sentry-release=Unknown"},"correlation_id":"01M3CMRXBM0SVV3D8GDBMBN65N","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:01ff7c5e2852721e05ff5756c6d4a33b9080a1ea2814ffefeacf179855b02d6e","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:06.450Z","job_size_bytes":467,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-e429ae47d80ab5a7c3ecd5eb: done: 0.674157 sec","job_status":"done","queue_duration_s":2.238829,"scheduling_latency_s":2.238851,"gitaly_calls":7,"gitaly_duration_s":0.066452,"redis_calls":33,"redis_duration_s":0.006361,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.00046,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00094,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.004065,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000896,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.585728,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.674157,"completed_at":"2026-09-25T16:01:09.363Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.022291,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1999gitlab # [ 393.523048] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:09.365Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"8e935b0f-02f8-459d-a26a-8b657e5cc619\"}"],"class":"WebHookWorker","jid":"20f808399b5709d680d573a2","created_at":"2026-09-25T16:01:06.468Z","trace_propagation_headers":{"sentry-trace":"6651615a0dd94a6783c3368585113dde-9521ce2410954fc7","baggage":"sentry-trace_id=6651615a0dd94a6783c3368585113dde,sentry-release=Unknown"},"correlation_id":"01M3CMRXBM0SVV3D8GDBMBN65N","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:9c76c4e34a8b09755063df95904895b4c91fe2865ca72885649d6a2c90ee93a5","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:06.470Z","job_size_bytes":1749,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-20f808399b5709d680d573a2: start","job_status":"start","queue_duration_s":2.895259,"scheduling_latency_s":2.89529}2000gitlab # [ 393.541190] nixbot[3401]: INFO: ::ffff:127.0.0.1:59790 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted2001gitlab # [ 393.542638] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:09.417Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"8e935b0f-02f8-459d-a26a-8b657e5cc619\"}"],"class":"WebHookWorker","jid":"20f808399b5709d680d573a2","created_at":"2026-09-25T16:01:06.468Z","trace_propagation_headers":{"sentry-trace":"6651615a0dd94a6783c3368585113dde-9521ce2410954fc7","baggage":"sentry-trace_id=6651615a0dd94a6783c3368585113dde,sentry-release=Unknown"},"correlation_id":"01M3CMRXBM0SVV3D8GDBMBN65N","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:9c76c4e34a8b09755063df95904895b4c91fe2865ca72885649d6a2c90ee93a5","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:06.470Z","job_size_bytes":1749,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-20f808399b5709d680d573a2: done: 0.051609 sec","job_status":"done","queue_duration_s":2.895259,"scheduling_latency_s":2.89529,"redis_calls":14,"redis_duration_s":0.003315,"redis_read_bytes":25,"redis_write_bytes":4685,"redis_queues_calls":2,"redis_queues_duration_s":0.000345,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3729,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001882,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001088,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.006447360000038316,"cpu_s":0.036029,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.051609,"completed_at":"2026-09-25T16:01:09.417Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002281,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2002gitlab # [ 393.591357] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:09.419Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"489a7da698d45dbdf60f40eb","created_at":"2026-09-25T16:01:06.585Z","trace_propagation_headers":{"sentry-trace":"1889fbea594747379be5acfaf38873cf-4542b069e49b4c48","baggage":"sentry-trace_id=1889fbea594747379be5acfaf38873cf,sentry-release=Unknown"},"correlation_id":"01M3CMPE07CPY3RA4RTWHTBTE3","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:06.590Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-489a7da698d45dbdf60f40eb: start","job_status":"start","queue_duration_s":2.829463,"scheduling_latency_s":2.82949}2003gitlab # [ 393.607894] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:09.490Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"489a7da698d45dbdf60f40eb","created_at":"2026-09-25T16:01:06.585Z","trace_propagation_headers":{"sentry-trace":"1889fbea594747379be5acfaf38873cf-4542b069e49b4c48","baggage":"sentry-trace_id=1889fbea594747379be5acfaf38873cf,sentry-release=Unknown"},"correlation_id":"01M3CMPE07CPY3RA4RTWHTBTE3","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:06.590Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-489a7da698d45dbdf60f40eb: done: 0.071359 sec","job_status":"done","queue_duration_s":2.829463,"scheduling_latency_s":2.82949,"redis_calls":19,"redis_duration_s":0.004118,"redis_read_bytes":21,"redis_write_bytes":4292,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000873,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000529,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2263,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002071,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000645,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.049159,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.010568,"duration_s":0.071359,"completed_at":"2026-09-25T16:01:09.490Z","load_balancing_strategy":"primary","db_duration_s":0.008535,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2004gitlab # [ 393.654479] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:09.492Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","5f36a1e1b0a71faf0174bb12"],"class":"WebHooks::LogExecutionWorker","jid":"9759e2896e3f79187a863589","created_at":"2026-09-25T16:01:07.824Z","trace_propagation_headers":{"sentry-trace":"1889fbea594747379be5acfaf38873cf-32d3e0e87b9240c9","baggage":"sentry-trace_id=1889fbea594747379be5acfaf38873cf,sentry-release=Unknown"},"correlation_id":"01M3CMPE07CPY3RA4RTWHTBTE3","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c0a1be8afab3005b17b64cba3c809d4cf1d350e3acaa9350107425222ea481b3","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:07.833Z","job_size_bytes":2593,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-9759e2896e3f79187a863589: start","job_status":"start","queue_duration_s":1.658868,"scheduling_latency_s":1.65889}2005gitlab # [ 393.671982] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:09.549Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","5f36a1e1b0a71faf0174bb12"],"class":"WebHooks::LogExecutionWorker","jid":"9759e2896e3f79187a863589","created_at":"2026-09-25T16:01:07.824Z","trace_propagation_headers":{"sentry-trace":"1889fbea594747379be5acfaf38873cf-32d3e0e87b9240c9","baggage":"sentry-trace_id=1889fbea594747379be5acfaf38873cf,sentry-release=Unknown"},"correlation_id":"01M3CMPE07CPY3RA4RTWHTBTE3","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c0a1be8afab3005b17b64cba3c809d4cf1d350e3acaa9350107425222ea481b3","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:07.833Z","job_size_bytes":2593,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-9759e2896e3f79187a863589: done: 0.05746 sec","job_status":"done","queue_duration_s":1.658868,"scheduling_latency_s":1.65889,"redis_calls":10,"redis_duration_s":0.0018700000000000001,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00055,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000979,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000341,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.007,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036206,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00033878,"duration_s":0.05746,"completed_at":"2026-09-25T16:01:09.549Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017901,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2006gitlab # [ 393.719288] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:09.551Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"183d9f980ad72cfe8c2ca321","created_at":"2026-09-25T16:01:08.370Z","trace_propagation_headers":{"sentry-trace":"cdc268062f104fc09074d5fa1bc39935-a6bea3b6c4bc44f2","baggage":"sentry-trace_id=cdc268062f104fc09074d5fa1bc39935,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3CMS3JH6R05MZ1PDB193R9V","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:08.374Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-183d9f980ad72cfe8c2ca321: start","job_status":"start","queue_duration_s":1.176534,"scheduling_latency_s":1.176553}2007gitlab # [ 393.736503] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:16:01:09 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 862008gitlab # [ 393.739860] gitlab-workhorse[821]: time="2026-09-25T16:01:09Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672009gitlab # [ 393.744369] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:16:01:09 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 792010gitlab # [ 393.747630] gitaly[914]: time="2026-09-25T16:01:09.516Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M3CMS4SPBX2N7P2Y412JNPHS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T16:01:09.506" grpc.time_ms=10.37 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc2011gitlab # [ 393.767674] gitaly[914]: time="2026-09-25T16:01:09.597Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3CMS4WEAWTFGM70AD1BFVMQ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T16:01:09.586" pid=914 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2012gitlab # [ 393.780926] gitaly[914]: time="2026-09-25T16:01:09.597Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMS4WEAWTFGM70AD1BFVMQ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T16:01:09.586" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc2013gitlab # [ 393.796207] gitaly[914]: time="2026-09-25T16:01:09.597Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMS4WEAWTFGM70AD1BFVMQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T16:01:09.586" grpc.time_ms=10.827 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc2014gitlab # [ 393.811053] gitaly[914]: time="2026-09-25T16:01:09.675Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3CMS3JH6R05MZ1PDB193R9V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-25T16:01:09.675" grpc.time_ms=0.499 pid=914 span.kind=server system=grpc2015gitlab # [ 393.821094] gitaly[914]: time="2026-09-25T16:01:09.698Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3CMS3JH6R05MZ1PDB193R9V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-25T16:01:09.697" grpc.time_ms=0.343 pid=914 span.kind=server system=grpc2016gitlab # [ 393.830951] gitaly[914]: time="2026-09-25T16:01:09.711Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M3CMS509HQCHPEV0G70ZN4GQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T16:01:09.697" grpc.time_ms=14.019 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc2017gitlab # [ 393.845043] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:16:01:09 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 702018gitlab # [ 393.847319] gitaly[914]: time="2026-09-25T16:01:09.725Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3CMS3JH6R05MZ1PDB193R9V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-25T16:01:09.724" grpc.time_ms=0.752 pid=914 span.kind=server system=grpc2019gitlab # [ 393.858722] gitaly[914]: time="2026-09-25T16:01:09.734Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3CMS3JH6R05MZ1PDB193R9V diskcache=a38b3c09-d653-445c-8c9f-27f2adcddb72 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-25T16:01:39.550" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-25T16:01:09.651" pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2020gitlab # [ 393.875165] gitaly[914]: time="2026-09-25T16:01:09.735Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=78 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7156 command.oublock=0 command.real_time_ms=80 command.spawn_token_fork_ms=0 command.system_time_ms=37 command.user_time_ms=41 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMS3JH6R05MZ1PDB193R9V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-25T16:01:39.550" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-25T16:01:09.651" grpc.time_ms=83.586 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2021gitlab # [ 393.896504] gitaly[914]: time="2026-09-25T16:01:09.787Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3CMS52HNZ96MAT65VG28ZQD grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T16:01:09.771" pid=914 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc2022gitlab # [ 393.907487] gitaly[914]: time="2026-09-25T16:01:09.787Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1103 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMS52HNZ96MAT65VG28ZQD grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T16:01:09.771" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc2023gitlab # [ 393.922170] gitlab-workhorse[821]: time="2026-09-25T16:01:09Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2162024gitlab # [ 393.924179] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:16:01:09 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 742025gitlab # [ 393.927073] gitaly[914]: time="2026-09-25T16:01:09.787Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1103 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMS52HNZ96MAT65VG28ZQD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-25T16:01:09.771" grpc.time_ms=15.936 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc2026gitlab # [ 393.942525] gitaly[914]: time="2026-09-25T16:01:09.791Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3CMS3JH6R05MZ1PDB193R9V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-25T16:01:39.689" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:09.790" grpc.time_ms=1.014 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2027gitlab # [ 393.956484] gitaly[914]: time="2026-09-25T16:01:09.828Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMS3JH6R05MZ1PDB193R9V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-25T16:01:39.727" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:09.827" grpc.time_ms=0.581 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2028gitlab # [ 393.970200] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:09.837Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"183d9f980ad72cfe8c2ca321","created_at":"2026-09-25T16:01:08.370Z","trace_propagation_headers":{"sentry-trace":"cdc268062f104fc09074d5fa1bc39935-a6bea3b6c4bc44f2","baggage":"sentry-trace_id=cdc268062f104fc09074d5fa1bc39935,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3CMS3JH6R05MZ1PDB193R9V","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:08.374Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":3274,"job_status":"rescheduled","message":"PipelineProcessWorker JID-183d9f980ad72cfe8c2ca321: rescheduled"}2029gitlab # [ 393.986200] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:09.846Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"183d9f980ad72cfe8c2ca321","created_at":"2026-09-25T16:01:08.370Z","trace_propagation_headers":{"sentry-trace":"cdc268062f104fc09074d5fa1bc39935-a6bea3b6c4bc44f2","baggage":"sentry-trace_id=cdc268062f104fc09074d5fa1bc39935,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3CMS3JH6R05MZ1PDB193R9V","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:08.374Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-183d9f980ad72cfe8c2ca321: done: 0.294879 sec","job_status":"done","queue_duration_s":1.176534,"scheduling_latency_s":1.176553,"gitaly_calls":3,"gitaly_duration_s":0.091345,"redis_calls":66,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.011723,"redis_read_bytes":148,"redis_write_bytes":18012,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000732,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000767,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.003094,"redis_queues_read_bytes":26,"redis_queues_write_bytes":12311,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.003732,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000256,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001047,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001363,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.045,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.012,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.012,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.023,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.155531,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.25118904,"duration_s":0.294879,"completed_at":"2026-09-25T16:01:09.846Z","load_balancing_strategy":"primary","db_duration_s":0.050231,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2030gitlab # [ 394.040524] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:09.847Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","20f808399b5709d680d573a2"],"class":"WebHooks::LogExecutionWorker","jid":"e4d39b4d23d08ee0519b087f","created_at":"2026-09-25T16:01:09.405Z","trace_propagation_headers":{"sentry-trace":"6651615a0dd94a6783c3368585113dde-169f476ff2bd4cf4","baggage":"sentry-trace_id=6651615a0dd94a6783c3368585113dde,sentry-release=Unknown"},"correlation_id":"01M3CMRXBM0SVV3D8GDBMBN65N","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e9ddbb94655ee6cf8f415203051cbb0258a24170db3208ce1c323e95a4c7d288","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:09.408Z","job_size_bytes":2592,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-e4d39b4d23d08ee0519b087f: start","job_status":"start","queue_duration_s":0.439598,"scheduling_latency_s":0.439624}2031gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=53d1e523bb688b08820df8ef6496a15b07ee0ccf'2032gitlab # [ 394.058710] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:09.876Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","20f808399b5709d680d573a2"],"class":"WebHooks::LogExecutionWorker","jid":"e4d39b4d23d08ee0519b087f","created_at":"2026-09-25T16:01:09.405Z","trace_propagation_headers":{"sentry-trace":"6651615a0dd94a6783c3368585113dde-169f476ff2bd4cf4","baggage":"sentry-trace_id=6651615a0dd94a6783c3368585113dde,sentry-release=Unknown"},"correlation_id":"01M3CMRXBM0SVV3D8GDBMBN65N","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e9ddbb94655ee6cf8f415203051cbb0258a24170db3208ce1c323e95a4c7d288","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:09.408Z","job_size_bytes":2592,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-e4d39b4d23d08ee0519b087f: done: 0.028787 sec","job_status":"done","queue_duration_s":0.439598,"scheduling_latency_s":0.439624,"redis_calls":7,"redis_duration_s":0.001745,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001274,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000471,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020479,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028787,"completed_at":"2026-09-25T16:01:09.876Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003216,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2033gitlab # [ 394.102906] gitaly[914]: time="2026-09-25T16:01:09.871Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMS55SSFT3KVJECXHC13F7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-25T16:01:39.070" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:09.870" grpc.time_ms=0.646 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2034gitlab # [ 394.116332] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:09.878Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"f39ac8ee2cf0f4a8b5821286","created_at":"2026-09-25T16:01:09.472Z","trace_propagation_headers":{"sentry-trace":"1889fbea594747379be5acfaf38873cf-cba2b7d781254e57","baggage":"sentry-trace_id=1889fbea594747379be5acfaf38873cf,sentry-release=Unknown"},"correlation_id":"01M3CMPE07CPY3RA4RTWHTBTE3","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:09.480Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-f39ac8ee2cf0f4a8b5821286: start","job_status":"start","queue_duration_s":0.397913,"scheduling_latency_s":0.397935}2035gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=53d1e523bb688b08820df8ef6496a15b07ee0ccf', in 0.06 seconds)2036(finished: subtest: push delivers the webhook, in 86.14 seconds)2037subtest: eval and build post commit statuses2038gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/53d1e523bb688b08820df8ef6496a15b07ee0ccf/statuses2039gitlab # [ 394.135527] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:09.968Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"f39ac8ee2cf0f4a8b5821286","created_at":"2026-09-25T16:01:09.472Z","trace_propagation_headers":{"sentry-trace":"1889fbea594747379be5acfaf38873cf-cba2b7d781254e57","baggage":"sentry-trace_id=1889fbea594747379be5acfaf38873cf,sentry-release=Unknown"},"correlation_id":"01M3CMPE07CPY3RA4RTWHTBTE3","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:09.480Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-f39ac8ee2cf0f4a8b5821286: done: 0.090477 sec","job_status":"done","queue_duration_s":0.397913,"scheduling_latency_s":0.397935,"redis_calls":11,"redis_duration_s":0.001869,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000441,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001094,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000334,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.070165,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00022616,"duration_s":0.090477,"completed_at":"2026-09-25T16:01:09.968Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015579,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2040gitlab # [ 394.184610] gitaly[914]: time="2026-09-25T16:01:09.891Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=591 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M3CMS55SSFT3KVJECXHC13F7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-25T16:01:39.080" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-25T16:01:09.880" grpc.time_ms=10.696 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2041gitlab # [ 394.199857] nixbot[3401]: INFO: ::ffff:127.0.0.1:59800 - "GET /api/repos/gitlab/root/test-flake/builds?commit=53d1e523bb688b08820df8ef6496a15b07ee0ccf HTTP/1.1" 200 OK2042gitlab # [ 394.202028] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:09.970Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"379f0b08b51fd5c37d122e48","created_at":"2026-09-25T16:01:09.642Z","trace_propagation_headers":{"sentry-trace":"cdc268062f104fc09074d5fa1bc39935-09bd453da9e44340","baggage":"sentry-trace_id=cdc268062f104fc09074d5fa1bc39935,sentry-release=Unknown"},"correlation_id":"01M3CMS3JH6R05MZ1PDB193R9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:09.645Z","job_size_bytes":23,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-379f0b08b51fd5c37d122e48: start","job_status":"start","queue_duration_s":0.324866,"scheduling_latency_s":0.324885}2043gitlab # [ 394.219198] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:10.023Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"379f0b08b51fd5c37d122e48","created_at":"2026-09-25T16:01:09.642Z","trace_propagation_headers":{"sentry-trace":"cdc268062f104fc09074d5fa1bc39935-09bd453da9e44340","baggage":"sentry-trace_id=cdc268062f104fc09074d5fa1bc39935,sentry-release=Unknown"},"correlation_id":"01M3CMS3JH6R05MZ1PDB193R9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:09.645Z","job_size_bytes":23,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-379f0b08b51fd5c37d122e48: done: 0.053798 sec","job_status":"done","queue_duration_s":0.324866,"scheduling_latency_s":0.324885,"redis_calls":7,"redis_duration_s":0.0031019999999999997,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001976,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001126,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032215,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.053798,"completed_at":"2026-09-25T16:01:10.023Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016771,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2044gitlab # [ 394.263402] gitaly[914]: time="2026-09-25T16:01:09.907Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M3CMS55SSFT3KVJECXHC13F7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-25T16:01:19.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-25T16:01:09.896" grpc.time_ms=10.426 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2045gitlab # [ 394.277421] nixbot[3401]: {"ts": "2026-09-25T16:01:10.108Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/53d1e523bb688b08820df8ef6496a15b07ee0ccf \"HTTP/1.1 201 Created\""}2046gitlab # [ 394.280188] nixbot[3401]: {"ts": "2026-09-25T16:01:10.119Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-53d1e523bb68-5"}2047gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/53d1e523bb688b08820df8ef6496a15b07ee0ccf/statuses, in 0.15 seconds)2048gitlab # [ 394.282506] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:10.027Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"e8cd02312632c62611c6d9ad","created_at":"2026-09-25T16:01:09.739Z","trace_propagation_headers":{"sentry-trace":"cdc268062f104fc09074d5fa1bc39935-09bd453da9e44340","baggage":"sentry-trace_id=cdc268062f104fc09074d5fa1bc39935,sentry-release=Unknown"},"correlation_id":"01M3CMS3JH6R05MZ1PDB193R9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:09.742Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-e8cd02312632c62611c6d9ad: start","job_status":"start","queue_duration_s":0.284989,"scheduling_latency_s":0.285008}2049gitlab # [ 394.300503] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:10.073Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"e8cd02312632c62611c6d9ad","created_at":"2026-09-25T16:01:09.739Z","trace_propagation_headers":{"sentry-trace":"cdc268062f104fc09074d5fa1bc39935-09bd453da9e44340","baggage":"sentry-trace_id=cdc268062f104fc09074d5fa1bc39935,sentry-release=Unknown"},"correlation_id":"01M3CMS3JH6R05MZ1PDB193R9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:09.742Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-e8cd02312632c62611c6d9ad: done: 0.046106 sec","job_status":"done","queue_duration_s":0.284989,"scheduling_latency_s":0.285008,"redis_calls":7,"redis_duration_s":0.0057350000000000005,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004974,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000761,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023864,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.046106,"completed_at":"2026-09-25T16:01:10.073Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012725,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2050gitlab # [ 394.347551] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:16:01:10 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/53d1e523bb688b08820df8ef6496a15b07ee0ccf HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 2902051gitlab # [ 394.350405] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:16:01:10 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/53d1e523bb688b08820df8ef6496a15b07ee0ccf/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1052052gitlab # [ 394.353342] gitaly[914]: time="2026-09-25T16:01:09.925Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3CMS55SSFT3KVJECXHC13F7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-25T16:01:19.034" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-25T16:01:09.924" grpc.time_ms=0.243 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2053gitlab # [ 394.364696] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:10.075Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"6b0aee1be00af199d6a40681","created_at":"2026-09-25T16:01:09.749Z","trace_propagation_headers":{"sentry-trace":"cdc268062f104fc09074d5fa1bc39935-09bd453da9e44340","baggage":"sentry-trace_id=cdc268062f104fc09074d5fa1bc39935,sentry-release=Unknown"},"correlation_id":"01M3CMS3JH6R05MZ1PDB193R9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:09.752Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-6b0aee1be00af199d6a40681: start","job_status":"start","queue_duration_s":0.323171,"scheduling_latency_s":0.323192}2054gitlab # [ 394.381543] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:10.115Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"6b0aee1be00af199d6a40681","created_at":"2026-09-25T16:01:09.749Z","trace_propagation_headers":{"sentry-trace":"cdc268062f104fc09074d5fa1bc39935-09bd453da9e44340","baggage":"sentry-trace_id=cdc268062f104fc09074d5fa1bc39935,sentry-release=Unknown"},"correlation_id":"01M3CMS3JH6R05MZ1PDB193R9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:09.752Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-6b0aee1be00af199d6a40681: done: 0.040099 sec","job_status":"done","queue_duration_s":0.323171,"scheduling_latency_s":0.323192,"redis_calls":7,"redis_duration_s":0.003738,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001282,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002456,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.011,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023187,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.040099,"completed_at":"2026-09-25T16:01:10.115Z","load_balancing_strategy":"primary","db_duration_s":0.010762,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2055gitlab # [ 394.424764] nixbot[3401]: {"ts": "2026-09-25T16:01:10.246Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-53d1e523bb68-5", "argv0": "bwrap"}2056gitlab # [ 394.427374] gitaly[914]: time="2026-09-25T16:01:09.938Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMS55SSFT3KVJECXHC13F7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-25T16:01:19.040" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-25T16:01:09.931" grpc.time_ms=6.915 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2057gitlab # [ 394.441096] gitaly[914]: time="2026-09-25T16:01:09.977Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3CMS55SSFT3KVJECXHC13F7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4154 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-25T16:01:09.976" grpc.time_ms=0.36 pid=914 span.kind=server system=grpc2058gitlab # [ 394.451112] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:10.116Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"dc05019e4ea796143a4371f7","created_at":"2026-09-25T16:01:09.762Z","trace_propagation_headers":{"sentry-trace":"cdc268062f104fc09074d5fa1bc39935-09bd453da9e44340","baggage":"sentry-trace_id=cdc268062f104fc09074d5fa1bc39935,sentry-release=Unknown"},"correlation_id":"01M3CMS3JH6R05MZ1PDB193R9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:09.764Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-dc05019e4ea796143a4371f7: start","job_status":"start","queue_duration_s":0.352799,"scheduling_latency_s":0.352818}2059gitlab # [ 394.467273] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:10.157Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"dc05019e4ea796143a4371f7","created_at":"2026-09-25T16:01:09.762Z","trace_propagation_headers":{"sentry-trace":"cdc268062f104fc09074d5fa1bc39935-09bd453da9e44340","baggage":"sentry-trace_id=cdc268062f104fc09074d5fa1bc39935,sentry-release=Unknown"},"correlation_id":"01M3CMS3JH6R05MZ1PDB193R9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:09.764Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-dc05019e4ea796143a4371f7: done: 0.040072 sec","job_status":"done","queue_duration_s":0.352799,"scheduling_latency_s":0.352818,"redis_calls":10,"redis_duration_s":0.006574,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003802,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.002372,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0004,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025953,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.040072,"completed_at":"2026-09-25T16:01:10.156Z","load_balancing_strategy":"primary","db_duration_s":0.005736,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2060gitlab # [ 394.512581] gitaly[914]: time="2026-09-25T16:01:10.003Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3CMS55SSFT3KVJECXHC13F7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4156 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-25T16:01:10.002" grpc.time_ms=0.469 pid=914 span.kind=server system=grpc2061gitlab # [ 394.522583] gitaly[914]: time="2026-09-25T16:01:10.013Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3CMS55SSFT3KVJECXHC13F7 diskcache=18d7cbf4-8910-40fc-be3c-98892162eae4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-25T16:01:19.049" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-25T16:01:09.940" pid=914 ref=refs/keep-around/53d1e523bb688b08820df8ef6496a15b07ee0ccf remote_ip="2001:db8:1::1" revision=53d1e523bb688b08820df8ef6496a15b07ee0ccf span.kind=server system=grpc username=root2062gitlab # [ 394.535043] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:10.158Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"75c7040e1af1f8a69a2e954a","created_at":"2026-09-25T16:01:09.764Z","trace_propagation_headers":{"sentry-trace":"cdc268062f104fc09074d5fa1bc39935-09bd453da9e44340","baggage":"sentry-trace_id=cdc268062f104fc09074d5fa1bc39935,sentry-release=Unknown"},"correlation_id":"01M3CMS3JH6R05MZ1PDB193R9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:09.766Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-75c7040e1af1f8a69a2e954a: start","job_status":"start","queue_duration_s":0.391979,"scheduling_latency_s":0.392002}2063gitlab # [ 394.554173] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:10.196Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"75c7040e1af1f8a69a2e954a","created_at":"2026-09-25T16:01:09.764Z","trace_propagation_headers":{"sentry-trace":"cdc268062f104fc09074d5fa1bc39935-09bd453da9e44340","baggage":"sentry-trace_id=cdc268062f104fc09074d5fa1bc39935,sentry-release=Unknown"},"correlation_id":"01M3CMS3JH6R05MZ1PDB193R9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:09.766Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-75c7040e1af1f8a69a2e954a: done: 0.038132 sec","job_status":"done","queue_duration_s":0.391979,"scheduling_latency_s":0.392002,"redis_calls":7,"redis_duration_s":0.002606,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001355,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001251,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020996,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.038132,"completed_at":"2026-09-25T16:01:10.196Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008795,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2064gitlab # [ 394.598371] gitaly[914]: time="2026-09-25T16:01:10.014Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=60 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5019 command.oublock=8 command.real_time_ms=69 command.spawn_token_fork_ms=0 command.system_time_ms=31 command.user_time_ms=29 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMS55SSFT3KVJECXHC13F7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-25T16:01:19.049" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-25T16:01:09.940" grpc.time_ms=73.685 pid=914 ref=refs/keep-around/53d1e523bb688b08820df8ef6496a15b07ee0ccf remote_ip="2001:db8:1::1" revision=53d1e523bb688b08820df8ef6496a15b07ee0ccf span.kind=server system=grpc username=root2065gitlab # [ 394.614360] nix-daemon[3564]: accepted connection from pid 4612, user nixbot2066gitlab # [ 394.615369] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:10.198Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"20145cd1b48ff7084c6ed942","created_at":"2026-09-25T16:01:09.767Z","trace_propagation_headers":{"sentry-trace":"cdc268062f104fc09074d5fa1bc39935-09bd453da9e44340","baggage":"sentry-trace_id=cdc268062f104fc09074d5fa1bc39935,sentry-release=Unknown"},"correlation_id":"01M3CMS3JH6R05MZ1PDB193R9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:09.769Z","job_size_bytes":19,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-20145cd1b48ff7084c6ed942: start","job_status":"start","queue_duration_s":0.429073,"scheduling_latency_s":0.429098}2067gitlab # [ 394.632313] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:10.228Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"20145cd1b48ff7084c6ed942","created_at":"2026-09-25T16:01:09.767Z","trace_propagation_headers":{"sentry-trace":"cdc268062f104fc09074d5fa1bc39935-09bd453da9e44340","baggage":"sentry-trace_id=cdc268062f104fc09074d5fa1bc39935,sentry-release=Unknown"},"correlation_id":"01M3CMS3JH6R05MZ1PDB193R9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:09.769Z","job_size_bytes":19,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-20145cd1b48ff7084c6ed942: done: 0.030091 sec","job_status":"done","queue_duration_s":0.429073,"scheduling_latency_s":0.429098,"redis_calls":8,"redis_duration_s":0.001535,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001192,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000343,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020821,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030091,"completed_at":"2026-09-25T16:01:10.228Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003512,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2068gitlab # [ 394.675504] gitaly[914]: time="2026-09-25T16:01:10.017Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMS55SSFT3KVJECXHC13F7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-25T16:01:40.017" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:10.017" grpc.time_ms=0.429 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2069gitlab # [ 394.688993] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:10.230Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"2080bdaacbe941fcac7c533f","created_at":"2026-09-25T16:01:09.770Z","trace_propagation_headers":{"sentry-trace":"cdc268062f104fc09074d5fa1bc39935-09bd453da9e44340","baggage":"sentry-trace_id=cdc268062f104fc09074d5fa1bc39935,sentry-release=Unknown"},"correlation_id":"01M3CMS3JH6R05MZ1PDB193R9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:09.772Z","job_size_bytes":66,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-2080bdaacbe941fcac7c533f: start","job_status":"start","queue_duration_s":0.458172,"scheduling_latency_s":0.458204}2070gitlab # [ 394.707753] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:10.259Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"2080bdaacbe941fcac7c533f","created_at":"2026-09-25T16:01:09.770Z","trace_propagation_headers":{"sentry-trace":"cdc268062f104fc09074d5fa1bc39935-09bd453da9e44340","baggage":"sentry-trace_id=cdc268062f104fc09074d5fa1bc39935,sentry-release=Unknown"},"correlation_id":"01M3CMS3JH6R05MZ1PDB193R9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:09.772Z","job_size_bytes":66,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-2080bdaacbe941fcac7c533f: done: 0.029063 sec","job_status":"done","queue_duration_s":0.458172,"scheduling_latency_s":0.458204,"redis_calls":7,"redis_duration_s":0.0015300000000000001,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001085,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000445,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020945,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029063,"completed_at":"2026-09-25T16:01:10.259Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002824,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2071gitlab # [ 394.755419] gitaly[914]: time="2026-09-25T16:01:10.150Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMS5DWSVA6FNCJ1EAG9V4G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-25T16:01:40.149" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:10.150" grpc.time_ms=0.632 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2072gitlab # [ 394.769357] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:10.261Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"d751fd9582a646530b04d186","created_at":"2026-09-25T16:01:09.772Z","trace_propagation_headers":{"sentry-trace":"cdc268062f104fc09074d5fa1bc39935-09bd453da9e44340","baggage":"sentry-trace_id=cdc268062f104fc09074d5fa1bc39935,sentry-release=Unknown"},"correlation_id":"01M3CMS3JH6R05MZ1PDB193R9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:09.775Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-d751fd9582a646530b04d186: start","job_status":"start","queue_duration_s":0.485853,"scheduling_latency_s":0.48588}2073gitlab # [ 394.786742] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:10.288Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"d751fd9582a646530b04d186","created_at":"2026-09-25T16:01:09.772Z","trace_propagation_headers":{"sentry-trace":"cdc268062f104fc09074d5fa1bc39935-09bd453da9e44340","baggage":"sentry-trace_id=cdc268062f104fc09074d5fa1bc39935,sentry-release=Unknown"},"correlation_id":"01M3CMS3JH6R05MZ1PDB193R9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:09.775Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-d751fd9582a646530b04d186: done: 0.027069 sec","job_status":"done","queue_duration_s":0.485853,"scheduling_latency_s":0.48588,"redis_calls":7,"redis_duration_s":0.001246,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00091,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000336,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018453,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027069,"completed_at":"2026-09-25T16:01:10.288Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00416,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2074gitlab # [ 394.831539] gitaly[914]: time="2026-09-25T16:01:10.523Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3CMS55SSFT3KVJECXHC13F7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-25T16:01:40.421" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:10.521" grpc.time_ms=0.968 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2075gitlab # [ 394.845449] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:10.289Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"43b09a0fad8a00c9419f4f63","created_at":"2026-09-25T16:01:09.775Z","trace_propagation_headers":{"sentry-trace":"cdc268062f104fc09074d5fa1bc39935-09bd453da9e44340","baggage":"sentry-trace_id=cdc268062f104fc09074d5fa1bc39935,sentry-release=Unknown"},"correlation_id":"01M3CMS3JH6R05MZ1PDB193R9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:09.777Z","job_size_bytes":6,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-43b09a0fad8a00c9419f4f63: start","job_status":"start","queue_duration_s":0.512002,"scheduling_latency_s":0.512024}2076gitlab # [ 394.862484] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:10.304Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"43b09a0fad8a00c9419f4f63","created_at":"2026-09-25T16:01:09.775Z","trace_propagation_headers":{"sentry-trace":"cdc268062f104fc09074d5fa1bc39935-09bd453da9e44340","baggage":"sentry-trace_id=cdc268062f104fc09074d5fa1bc39935,sentry-release=Unknown"},"correlation_id":"01M3CMS3JH6R05MZ1PDB193R9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:09.777Z","job_size_bytes":6,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-43b09a0fad8a00c9419f4f63: done: 0.014769 sec","job_status":"done","queue_duration_s":0.512002,"scheduling_latency_s":0.512024,"redis_calls":7,"redis_duration_s":0.00122,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000879,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000341,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009537,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014769,"completed_at":"2026-09-25T16:01:10.304Z","load_balancing_strategy":"primary","db_duration_s":0.000464,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2077gitlab # [ 394.907220] gitaly[914]: time="2026-09-25T16:01:10.561Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMS55SSFT3KVJECXHC13F7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-25T16:01:40.460" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:10.560" grpc.time_ms=0.622 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2078gitlab # [ 394.920513] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:10.305Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"d89fc87c6c110b7c9fb51b46","created_at":"2026-09-25T16:01:09.778Z","trace_propagation_headers":{"sentry-trace":"cdc268062f104fc09074d5fa1bc39935-09bd453da9e44340","baggage":"sentry-trace_id=cdc268062f104fc09074d5fa1bc39935,sentry-release=Unknown"},"correlation_id":"01M3CMS3JH6R05MZ1PDB193R9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:09.779Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-d89fc87c6c110b7c9fb51b46: start","job_status":"start","queue_duration_s":0.525912,"scheduling_latency_s":0.525934}2079gitlab # [ 394.936947] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:10.329Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"d89fc87c6c110b7c9fb51b46","created_at":"2026-09-25T16:01:09.778Z","trace_propagation_headers":{"sentry-trace":"cdc268062f104fc09074d5fa1bc39935-09bd453da9e44340","baggage":"sentry-trace_id=cdc268062f104fc09074d5fa1bc39935,sentry-release=Unknown"},"correlation_id":"01M3CMS3JH6R05MZ1PDB193R9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:09.779Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-d89fc87c6c110b7c9fb51b46: done: 0.023389 sec","job_status":"done","queue_duration_s":0.525912,"scheduling_latency_s":0.525934,"redis_calls":7,"redis_duration_s":0.001207,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00088,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000327,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015979,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023389,"completed_at":"2026-09-25T16:01:10.328Z","load_balancing_strategy":"primary","db_duration_s":0.003482,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2080gitlab # [ 394.982040] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:10.330Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"6b0ae69d7d01ef610a3fe263","created_at":"2026-09-25T16:01:09.838Z","trace_propagation_headers":{"sentry-trace":"cdc268062f104fc09074d5fa1bc39935-09bd453da9e44340","baggage":"sentry-trace_id=cdc268062f104fc09074d5fa1bc39935,sentry-release=Unknown"},"correlation_id":"01M3CMS3JH6R05MZ1PDB193R9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:09.840Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-6b0ae69d7d01ef610a3fe263: start","job_status":"start","queue_duration_s":0.489905,"scheduling_latency_s":0.489923}2081gitlab # [ 394.998679] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:10.361Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"6b0ae69d7d01ef610a3fe263","created_at":"2026-09-25T16:01:09.838Z","trace_propagation_headers":{"sentry-trace":"cdc268062f104fc09074d5fa1bc39935-09bd453da9e44340","baggage":"sentry-trace_id=cdc268062f104fc09074d5fa1bc39935,sentry-release=Unknown"},"correlation_id":"01M3CMS3JH6R05MZ1PDB193R9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:09.840Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-6b0ae69d7d01ef610a3fe263: done: 0.030897 sec","job_status":"done","queue_duration_s":0.489905,"scheduling_latency_s":0.489923,"redis_calls":8,"redis_duration_s":0.001566,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00115,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000416,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019083,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030897,"completed_at":"2026-09-25T16:01:10.361Z","load_balancing_strategy":"primary","db_duration_s":0.007934,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2082gitlab # [ 395.043287] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:10.362Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"fee4bd8c1a5e6a9d8264ef5d","created_at":"2026-09-25T16:01:10.019Z","trace_propagation_headers":{"sentry-trace":"f4a81a679461454ba4e26792af0bf7b9-1fa2e3bafb244771","baggage":"sentry-trace_id=f4a81a679461454ba4e26792af0bf7b9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3CMS55SSFT3KVJECXHC13F7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:10.023Z","job_size_bytes":65,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-fee4bd8c1a5e6a9d8264ef5d: start","job_status":"start","queue_duration_s":0.339455,"scheduling_latency_s":0.339473}2083gitlab # [ 395.061579] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:10.392Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"fee4bd8c1a5e6a9d8264ef5d","created_at":"2026-09-25T16:01:10.019Z","trace_propagation_headers":{"sentry-trace":"f4a81a679461454ba4e26792af0bf7b9-1fa2e3bafb244771","baggage":"sentry-trace_id=f4a81a679461454ba4e26792af0bf7b9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3CMS55SSFT3KVJECXHC13F7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:10.023Z","job_size_bytes":65,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-fee4bd8c1a5e6a9d8264ef5d: done: 0.029885 sec","job_status":"done","queue_duration_s":0.339455,"scheduling_latency_s":0.339473,"redis_calls":7,"redis_duration_s":0.0012980000000000001,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000853,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000445,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0207,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029885,"completed_at":"2026-09-25T16:01:10.392Z","load_balancing_strategy":"primary","db_duration_s":0.005254,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2084gitlab # [ 395.107146] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:10.393Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"8f613ed601d241863fdca5ff","created_at":"2026-09-25T16:01:10.023Z","trace_propagation_headers":{"sentry-trace":"f4a81a679461454ba4e26792af0bf7b9-1fa2e3bafb244771","baggage":"sentry-trace_id=f4a81a679461454ba4e26792af0bf7b9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3CMS55SSFT3KVJECXHC13F7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:10.030Z","job_size_bytes":65,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-8f613ed601d241863fdca5ff: start","job_status":"start","queue_duration_s":0.363384,"scheduling_latency_s":0.3634}2085gitlab # [ 395.125771] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:10.449Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"8f613ed601d241863fdca5ff","created_at":"2026-09-25T16:01:10.023Z","trace_propagation_headers":{"sentry-trace":"f4a81a679461454ba4e26792af0bf7b9-1fa2e3bafb244771","baggage":"sentry-trace_id=f4a81a679461454ba4e26792af0bf7b9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3CMS55SSFT3KVJECXHC13F7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:10.030Z","job_size_bytes":65,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-8f613ed601d241863fdca5ff: done: 0.056184 sec","job_status":"done","queue_duration_s":0.363384,"scheduling_latency_s":0.3634,"redis_calls":10,"redis_duration_s":0.001671,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000967,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.000704,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035379,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.056184,"completed_at":"2026-09-25T16:01:10.449Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016414,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2086gitlab # [ 395.175749] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:10.451Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"b827b69321688d89146bd4c8","created_at":"2026-09-25T16:01:10.090Z","trace_propagation_headers":{"sentry-trace":"f4a81a679461454ba4e26792af0bf7b9-1fa2e3bafb244771","baggage":"sentry-trace_id=f4a81a679461454ba4e26792af0bf7b9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3CMS55SSFT3KVJECXHC13F7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:10.092Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-b827b69321688d89146bd4c8: start","job_status":"start","queue_duration_s":0.358141,"scheduling_latency_s":0.358158}2087gitlab # [ 395.193209] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:10.571Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"b827b69321688d89146bd4c8","created_at":"2026-09-25T16:01:10.090Z","trace_propagation_headers":{"sentry-trace":"f4a81a679461454ba4e26792af0bf7b9-1fa2e3bafb244771","baggage":"sentry-trace_id=f4a81a679461454ba4e26792af0bf7b9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3CMS55SSFT3KVJECXHC13F7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:10.092Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-b827b69321688d89146bd4c8: done: 0.121017 sec","job_status":"done","queue_duration_s":0.358141,"scheduling_latency_s":0.358158,"gitaly_calls":2,"gitaly_duration_s":0.006299,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.004922,"redis_read_bytes":79,"redis_write_bytes":5122,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000715,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000894,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000198,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001174,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000169,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000377,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00068,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.004,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.092994,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.09196432,"duration_s":0.121017,"completed_at":"2026-09-25T16:01:10.571Z","load_balancing_strategy":"primary","db_duration_s":0.023081,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2088gitlab # [ 395.247923] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:10.573Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"d1a85bd09890eeab1c96d7d2","created_at":"2026-09-25T16:01:10.506Z","trace_propagation_headers":{"sentry-trace":"f4a81a679461454ba4e26792af0bf7b9-082b1bcc5d6e4801","baggage":"sentry-trace_id=f4a81a679461454ba4e26792af0bf7b9,sentry-release=Unknown"},"correlation_id":"01M3CMS55SSFT3KVJECXHC13F7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:10.508Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-d1a85bd09890eeab1c96d7d2: start","job_status":"start","queue_duration_s":0.064767,"scheduling_latency_s":0.064784}2089gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/53d1e523bb688b08820df8ef6496a15b07ee0ccf/statuses2090gitlab # [ 395.265156] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:10.606Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"d1a85bd09890eeab1c96d7d2","created_at":"2026-09-25T16:01:10.506Z","trace_propagation_headers":{"sentry-trace":"f4a81a679461454ba4e26792af0bf7b9-082b1bcc5d6e4801","baggage":"sentry-trace_id=f4a81a679461454ba4e26792af0bf7b9,sentry-release=Unknown"},"correlation_id":"01M3CMS55SSFT3KVJECXHC13F7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:10.508Z","job_size_bytes":3,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-d1a85bd09890eeab1c96d7d2: done: 0.032805 sec","job_status":"done","queue_duration_s":0.064767,"scheduling_latency_s":0.064784,"redis_calls":7,"redis_duration_s":0.0012309999999999999,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00086,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000371,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024408,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032805,"completed_at":"2026-09-25T16:01:10.605Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004935,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2091gitlab # [ 395.416921] gitaly[914]: time="2026-09-25T16:01:11.306Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMS6HZB4EBE5BC22KRC05F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-25T16:01:41.305" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:11.305" grpc.time_ms=0.867 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2092gitlab # [ 395.457907] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:16:01:11 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/53d1e523bb688b08820df8ef6496a15b07ee0ccf/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1162093gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/53d1e523bb688b08820df8ef6496a15b07ee0ccf/statuses, in 0.16 seconds)2094gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/53d1e523bb688b08820df8ef6496a15b07ee0ccf/statuses2095gitlab # [ 396.561030] gitaly[914]: time="2026-09-25T16:01:12.450Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMS7P1JR166ADFKAN2C1P8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-25T16:01:42.449" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:12.449" grpc.time_ms=0.987 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2096gitlab # [ 396.595269] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:16:01:12 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/53d1e523bb688b08820df8ef6496a15b07ee0ccf/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 992097gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/53d1e523bb688b08820df8ef6496a15b07ee0ccf/statuses, in 0.14 seconds)2098gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/53d1e523bb688b08820df8ef6496a15b07ee0ccf/statuses2099gitlab # [ 397.704866] gitaly[914]: time="2026-09-25T16:01:13.594Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMS8SK127CHNPZEYNFEE00 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-25T16:01:43.593" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:13.593" grpc.time_ms=0.882 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2100gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/53d1e523bb688b08820df8ef6496a15b07ee0ccf/statuses, in 0.14 seconds)2101gitlab # [ 397.740079] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:16:01:13 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/53d1e523bb688b08820df8ef6496a15b07ee0ccf/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1062102gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/53d1e523bb688b08820df8ef6496a15b07ee0ccf/statuses2103gitlab # [ 398.842604] nix-daemon[3564]: accepted connection from pid 4646, user nixbot2104gitlab # [ 398.844898] gitaly[914]: time="2026-09-25T16:01:14.735Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMS9X8KWWMR1SFQW5VBZ1C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-25T16:01:44.733" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:14.734" grpc.time_ms=0.838 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2105gitlab # [ 398.864418] nix-daemon[3564]: accepted connection from pid 4612, user nixbot2106gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/53d1e523bb688b08820df8ef6496a15b07ee0ccf/statuses, in 0.14 seconds)2107gitlab # [ 398.878971] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:16:01:14 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/53d1e523bb688b08820df8ef6496a15b07ee0ccf/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1042108gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/53d1e523bb688b08820df8ef6496a15b07ee0ccf/statuses2109gitlab # [ 399.972364] gitaly[914]: time="2026-09-25T16:01:15.862Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMSB0R6HMT6Y9FHE8746K3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-25T16:01:45.860" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:15.861" grpc.time_ms=0.791 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2110gitlab # [ 400.005341] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:16:01:15 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/53d1e523bb688b08820df8ef6496a15b07ee0ccf/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 932111gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/53d1e523bb688b08820df8ef6496a15b07ee0ccf/statuses, in 0.13 seconds)2112gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/53d1e523bb688b08820df8ef6496a15b07ee0ccf/statuses2113gitlab # [ 401.101721] gitaly[914]: time="2026-09-25T16:01:16.991Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMSC4227XX9DM3WGJJ0J43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-25T16:01:46.090" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:16.990" grpc.time_ms=0.904 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2114gitlab # [ 401.134750] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:16:01:17 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/53d1e523bb688b08820df8ef6496a15b07ee0ccf/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 942115gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/53d1e523bb688b08820df8ef6496a15b07ee0ccf/statuses, in 0.13 seconds)2116gitlab # [ 401.254597] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:17.144Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"4cdb7564709a0ab45bf979b6","created_at":"2026-09-25T16:01:17.095Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-7e86b06e1b2d4c83","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"690e4b79217e461d8906f081166f89e8","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.141Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-4cdb7564709a0ab45bf979b6: start","job_status":"start","queue_duration_s":0.002326,"scheduling_latency_s":0.002363}2117gitlab # [ 401.353882] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:17.243Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"4cdb7564709a0ab45bf979b6","created_at":"2026-09-25T16:01:17.095Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-7e86b06e1b2d4c83","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"690e4b79217e461d8906f081166f89e8","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.141Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-4cdb7564709a0ab45bf979b6: done: 0.099653 sec","job_status":"done","queue_duration_s":0.002326,"scheduling_latency_s":0.002363,"redis_calls":9,"redis_duration_s":0.010042,"redis_read_bytes":7,"redis_write_bytes":1183,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001741,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006923,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001378,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029776,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.099653,"completed_at":"2026-09-25T16:01:17.243Z","load_balancing_strategy":"primary","db_duration_s":0.02699,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2118gitlab # [ 401.419619] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:17.257Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"32fd6b0027a5a090856bbb2d","created_at":"2026-09-25T16:01:17.149Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-7e86b06e1b2d4c83","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.158Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-32fd6b0027a5a090856bbb2d: start","job_status":"start","queue_duration_s":0.098805,"scheduling_latency_s":0.098835}2119gitlab # [ 401.946011] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:17.835Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"32fd6b0027a5a090856bbb2d","created_at":"2026-09-25T16:01:17.149Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-7e86b06e1b2d4c83","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.158Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-32fd6b0027a5a090856bbb2d: done: 0.578958 sec","job_status":"done","queue_duration_s":0.098805,"scheduling_latency_s":0.098835,"redis_calls":70,"redis_duration_s":0.322142,"redis_read_bytes":118,"redis_write_bytes":42530,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.021723,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.00153,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.296072,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.002817,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.164118,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.578958,"completed_at":"2026-09-25T16:01:17.835Z","load_balancing_strategy":"primary","db_duration_s":0.008165,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2120gitlab # [ 402.013214] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:17.841Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"61fffb4612b8b80d93433479","created_at":"2026-09-25T16:01:17.173Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-7e86b06e1b2d4c83","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"24e248da82cc056e4d2a6aae59d24e69","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.182Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-61fffb4612b8b80d93433479: start","job_status":"start","queue_duration_s":0.658528,"scheduling_latency_s":0.658558}2121gitlab # [ 402.057355] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:17.947Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"61fffb4612b8b80d93433479","created_at":"2026-09-25T16:01:17.173Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-7e86b06e1b2d4c83","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"24e248da82cc056e4d2a6aae59d24e69","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.182Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-61fffb4612b8b80d93433479: done: 0.106578 sec","job_status":"done","queue_duration_s":0.658528,"scheduling_latency_s":0.658558,"redis_calls":18,"redis_duration_s":0.007887,"redis_read_bytes":20,"redis_write_bytes":4702,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001497,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000385,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.003568,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.002437,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.041,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.041,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.042227,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.106578,"completed_at":"2026-09-25T16:01:17.947Z","load_balancing_strategy":"primary","db_duration_s":0.04143,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2122gitlab # [ 402.113470] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:17.949Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"d861c58509aa4b259885bf2e","created_at":"2026-09-25T16:01:17.187Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-7e86b06e1b2d4c83","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d2a1848e3f96a91ee548657085ebe609","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.195Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-d861c58509aa4b259885bf2e: start","job_status":"start","queue_duration_s":0.754085,"scheduling_latency_s":0.754124}2123gitlab # [ 402.131629] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:17.964Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}2124gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/53d1e523bb688b08820df8ef6496a15b07ee0ccf/statuses2125gitlab # [ 402.135594] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:17.977Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"d861c58509aa4b259885bf2e","created_at":"2026-09-25T16:01:17.187Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-7e86b06e1b2d4c83","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d2a1848e3f96a91ee548657085ebe609","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.195Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-d861c58509aa4b259885bf2e: done: 0.028544 sec","job_status":"done","queue_duration_s":0.754085,"scheduling_latency_s":0.754124,"redis_calls":9,"redis_duration_s":0.002251,"redis_read_bytes":7,"redis_write_bytes":1185,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000417,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00141,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000424,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017028,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028544,"completed_at":"2026-09-25T16:01:17.977Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000663,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2126gitlab # [ 402.181451] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:17.979Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"da44763c3ff6504e2ddfbe0b","created_at":"2026-09-25T16:01:17.203Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-7e86b06e1b2d4c83","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"46fb6e2872df59daedd87c5a7e31fed5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.214Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-da44763c3ff6504e2ddfbe0b: start","job_status":"start","queue_duration_s":0.764588,"scheduling_latency_s":0.764611}2127gitlab # [ 402.196251] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:17.992Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}2128gitlab # [ 402.199773] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:18.002Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"da44763c3ff6504e2ddfbe0b","created_at":"2026-09-25T16:01:17.203Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-7e86b06e1b2d4c83","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"46fb6e2872df59daedd87c5a7e31fed5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.214Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-da44763c3ff6504e2ddfbe0b: done: 0.023343 sec","job_status":"done","queue_duration_s":0.764588,"scheduling_latency_s":0.764611,"redis_calls":9,"redis_duration_s":0.002801,"redis_read_bytes":7,"redis_write_bytes":1105,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000391,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001034,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001376,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014361,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023343,"completed_at":"2026-09-25T16:01:18.002Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000596,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2129gitlab # [ 402.243398] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:18.006Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"590519145b54dd0971b0a3c3","created_at":"2026-09-25T16:01:17.221Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-7e86b06e1b2d4c83","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a8c5b4af0375c3b0665f59273d7e6822","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.239Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-590519145b54dd0971b0a3c3: start","job_status":"start","queue_duration_s":0.767179,"scheduling_latency_s":0.767207}2130gitlab # [ 402.258953] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:18.022Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}2131gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/53d1e523bb688b08820df8ef6496a15b07ee0ccf/statuses, in 0.13 seconds)2132gitlab # [ 402.262952] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:18.037Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"590519145b54dd0971b0a3c3","created_at":"2026-09-25T16:01:17.221Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-7e86b06e1b2d4c83","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a8c5b4af0375c3b0665f59273d7e6822","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.239Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-590519145b54dd0971b0a3c3: done: 0.030308 sec","job_status":"done","queue_duration_s":0.767179,"scheduling_latency_s":0.767207,"redis_calls":9,"redis_duration_s":0.002236,"redis_read_bytes":7,"redis_write_bytes":1178,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000427,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001286,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000523,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01798,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030308,"completed_at":"2026-09-25T16:01:18.036Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000603,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2133gitlab # [ 402.308472] gitaly[914]: time="2026-09-25T16:01:18.125Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMSD7DFGSV6D2TVRSK6V9W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-25T16:01:48.124" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:18.124" grpc.time_ms=0.703 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2134gitlab # [ 402.322889] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:16:01:18 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/53d1e523bb688b08820df8ef6496a15b07ee0ccf/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 922135gitlab # [ 402.325681] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:18.039Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"a26a78dbe58e94b99c3aed86","created_at":"2026-09-25T16:01:17.253Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-7e86b06e1b2d4c83","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0075c72fab083a9f3ebd221ad33422a6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.262Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-a26a78dbe58e94b99c3aed86: start","job_status":"start","queue_duration_s":0.777135,"scheduling_latency_s":0.77716}2136gitlab # [ 402.340413] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:18.056Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}2137gitlab # [ 402.343622] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:18.069Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"a26a78dbe58e94b99c3aed86","created_at":"2026-09-25T16:01:17.253Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-7e86b06e1b2d4c83","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0075c72fab083a9f3ebd221ad33422a6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.262Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-a26a78dbe58e94b99c3aed86: done: 0.029822 sec","job_status":"done","queue_duration_s":0.777135,"scheduling_latency_s":0.77716,"redis_calls":9,"redis_duration_s":0.0032839999999999996,"redis_read_bytes":7,"redis_write_bytes":1112,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00049,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002338,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000456,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017887,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029822,"completed_at":"2026-09-25T16:01:18.069Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000713,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2138gitlab # [ 402.387318] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:18.071Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"a6b3a5d25c905fe91197dc92","created_at":"2026-09-25T16:01:17.270Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-7e86b06e1b2d4c83","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"95316174fee2e1b3f87ec4830bb938f3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.282Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-a6b3a5d25c905fe91197dc92: start","job_status":"start","queue_duration_s":0.789065,"scheduling_latency_s":0.789092}2139gitlab # [ 402.411698] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:18.161Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"a6b3a5d25c905fe91197dc92","created_at":"2026-09-25T16:01:17.270Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-7e86b06e1b2d4c83","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"95316174fee2e1b3f87ec4830bb938f3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.282Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-a6b3a5d25c905fe91197dc92: done: 0.090482 sec","job_status":"done","queue_duration_s":0.789065,"scheduling_latency_s":0.789092,"redis_calls":11,"redis_duration_s":0.0028339999999999997,"redis_read_bytes":9,"redis_write_bytes":1518,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000428,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001166,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00124,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.053606,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00093136,"exclusive_lock_hold_duration_s":0.04779844,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.090482,"completed_at":"2026-09-25T16:01:18.161Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.025672,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2140gitlab # [ 402.472137] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:18.163Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"5c8b453e3805201fc8c655e7","created_at":"2026-09-25T16:01:17.297Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-7e86b06e1b2d4c83","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d546c33caf6d0950218191c3864b84e8","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.311Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-5c8b453e3805201fc8c655e7: start","job_status":"start","queue_duration_s":0.852582,"scheduling_latency_s":0.852605}2141gitlab # [ 402.486775] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:18.199Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"5c8b453e3805201fc8c655e7","created_at":"2026-09-25T16:01:17.297Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-7e86b06e1b2d4c83","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d546c33caf6d0950218191c3864b84e8","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.311Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-5c8b453e3805201fc8c655e7: done: 0.035153 sec","job_status":"done","queue_duration_s":0.852582,"scheduling_latency_s":0.852605,"redis_calls":9,"redis_duration_s":0.002446,"redis_read_bytes":7,"redis_write_bytes":1026,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00043,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001215,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000801,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021572,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.035153,"completed_at":"2026-09-25T16:01:18.198Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00453,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2142gitlab # [ 402.529290] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:18.202Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"3bb6d3824437a3be3bb2df6b","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-7e86b06e1b2d4c83","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"327b289fa686adcf856627e0a54c6873","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.354Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-3bb6d3824437a3be3bb2df6b: start","job_status":"start","queue_duration_s":0.847902,"scheduling_latency_s":0.847928}2143gitlab # [ 402.545407] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:18.248Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"3bb6d3824437a3be3bb2df6b","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-7e86b06e1b2d4c83","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"327b289fa686adcf856627e0a54c6873","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.354Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-3bb6d3824437a3be3bb2df6b: done: 0.045815 sec","job_status":"done","queue_duration_s":0.847902,"scheduling_latency_s":0.847928,"redis_calls":12,"redis_duration_s":0.00281,"redis_read_bytes":10,"redis_write_bytes":1576,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000414,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00144,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000956,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031535,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01328246,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.045815,"completed_at":"2026-09-25T16:01:18.247Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003858,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2144gitlab # [ 402.590195] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:18.249Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"76bc1a46300fdcd4c25a6500","created_at":"2026-09-25T16:01:17.374Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-7e86b06e1b2d4c83","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fec52258d4d70a7ce343e4a579623841","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.386Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-76bc1a46300fdcd4c25a6500: start","job_status":"start","queue_duration_s":0.863283,"scheduling_latency_s":0.863306}2145gitlab # [ 402.605240] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:18.279Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"76bc1a46300fdcd4c25a6500","created_at":"2026-09-25T16:01:17.374Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-7e86b06e1b2d4c83","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fec52258d4d70a7ce343e4a579623841","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.386Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-76bc1a46300fdcd4c25a6500: done: 0.029669 sec","job_status":"done","queue_duration_s":0.863283,"scheduling_latency_s":0.863306,"redis_calls":11,"redis_duration_s":0.002418,"redis_read_bytes":9,"redis_write_bytes":1459,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000762,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001266,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00039,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017947,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029669,"completed_at":"2026-09-25T16:01:18.279Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001097,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2146gitlab # [ 402.648179] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:18.280Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"957a59bdf7d09fd8d8ab7707","created_at":"2026-09-25T16:01:17.432Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-7e86b06e1b2d4c83","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7031c9a5e977e9ada0a64d18cd5a4bbe","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-25T16:01:17.444Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-957a59bdf7d09fd8d8ab7707: start","job_status":"start","queue_duration_s":0.836308,"scheduling_latency_s":0.836326}2147gitlab # [ 402.661564] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:18.314Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"957a59bdf7d09fd8d8ab7707","created_at":"2026-09-25T16:01:17.432Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-7e86b06e1b2d4c83","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7031c9a5e977e9ada0a64d18cd5a4bbe","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-25T16:01:17.444Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-957a59bdf7d09fd8d8ab7707: done: 0.033976 sec","job_status":"done","queue_duration_s":0.836308,"scheduling_latency_s":0.836326,"redis_calls":9,"redis_duration_s":0.0014,"redis_read_bytes":7,"redis_write_bytes":1068,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00028,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000818,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000302,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021503,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033976,"completed_at":"2026-09-25T16:01:18.314Z","load_balancing_strategy":"primary","db_duration_s":0.004522,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2148gitlab # [ 402.702837] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:18.315Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"0c8e3a163370f3786db01bc5","created_at":"2026-09-25T16:01:17.462Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-7e86b06e1b2d4c83","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ab26ee13f56091161b96881cdcd5cdac","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-25T16:01:17.476Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-0c8e3a163370f3786db01bc5: start","job_status":"start","queue_duration_s":0.839365,"scheduling_latency_s":0.839383}2149gitlab # [ 402.716661] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:18.329Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}2150gitlab # [ 402.719950] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:18.339Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"0c8e3a163370f3786db01bc5","created_at":"2026-09-25T16:01:17.462Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-7e86b06e1b2d4c83","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ab26ee13f56091161b96881cdcd5cdac","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-25T16:01:17.476Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-0c8e3a163370f3786db01bc5: done: 0.023892 sec","job_status":"done","queue_duration_s":0.839365,"scheduling_latency_s":0.839383,"redis_calls":9,"redis_duration_s":0.001687,"redis_read_bytes":7,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000391,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000908,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000388,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011999,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023892,"completed_at":"2026-09-25T16:01:18.339Z","load_balancing_strategy":"primary","db_duration_s":0.000712,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2151gitlab # [ 402.762847] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:18.341Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"bb4f72d855ca57a2f42cf6fa","created_at":"2026-09-25T16:01:17.488Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-7e86b06e1b2d4c83","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cb8fe8d4072d485c73be478b41551587","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.500Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-bb4f72d855ca57a2f42cf6fa: start","job_status":"start","queue_duration_s":0.840473,"scheduling_latency_s":0.840493}2152gitlab # [ 402.777188] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:18.364Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"bb4f72d855ca57a2f42cf6fa","created_at":"2026-09-25T16:01:17.488Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-7e86b06e1b2d4c83","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cb8fe8d4072d485c73be478b41551587","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.500Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-bb4f72d855ca57a2f42cf6fa: done: 0.023743 sec","job_status":"done","queue_duration_s":0.840473,"scheduling_latency_s":0.840493,"redis_calls":9,"redis_duration_s":0.0015769999999999998,"redis_read_bytes":7,"redis_write_bytes":984,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000311,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000908,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000358,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014753,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023743,"completed_at":"2026-09-25T16:01:18.364Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002574,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2153gitlab # [ 402.819839] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:18.367Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"c1dff1b3960fa91845bbef0c","created_at":"2026-09-25T16:01:17.519Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-7e86b06e1b2d4c83","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6100d3fb8ac309ab075543fa030b9f34","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.533Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-c1dff1b3960fa91845bbef0c: start","job_status":"start","queue_duration_s":0.833341,"scheduling_latency_s":0.833358}2154gitlab # [ 402.834668] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:18.394Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"c1dff1b3960fa91845bbef0c","created_at":"2026-09-25T16:01:17.519Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-7e86b06e1b2d4c83","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6100d3fb8ac309ab075543fa030b9f34","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.533Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-c1dff1b3960fa91845bbef0c: done: 0.026868 sec","job_status":"done","queue_duration_s":0.833341,"scheduling_latency_s":0.833358,"redis_calls":11,"redis_duration_s":0.003022,"redis_read_bytes":9,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000316,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001995,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000711,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013618,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00047536,"exclusive_lock_hold_duration_s":0.00270242,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.026868,"completed_at":"2026-09-25T16:01:18.394Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001721,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2155gitlab # [ 402.878648] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:18.395Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"9f79a64bfc456c67344c832d","created_at":"2026-09-25T16:01:17.545Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-7e86b06e1b2d4c83","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2e1bb66b553f68439f1a2ff21ad84f80","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.561Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-9f79a64bfc456c67344c832d: start","job_status":"start","queue_duration_s":0.833655,"scheduling_latency_s":0.833672}2156gitlab # [ 402.893009] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:18.624Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"9f79a64bfc456c67344c832d","created_at":"2026-09-25T16:01:17.545Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-7e86b06e1b2d4c83","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2e1bb66b553f68439f1a2ff21ad84f80","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.561Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-9f79a64bfc456c67344c832d: done: 0.228816 sec","job_status":"done","queue_duration_s":0.833655,"scheduling_latency_s":0.833672,"redis_calls":13,"redis_duration_s":0.0022589999999999997,"redis_read_bytes":11,"redis_write_bytes":1563,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000611,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":576,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000874,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000774,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.033,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.032,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.186506,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00047338,"exclusive_lock_hold_duration_s":0.20664074,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.228816,"completed_at":"2026-09-25T16:01:18.624Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.048998,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2157gitlab # [ 402.938270] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:18.625Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"a5bac95fd7c6d5dc762f9d6e","created_at":"2026-09-25T16:01:17.579Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-7e86b06e1b2d4c83","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b7cacb52b16e8845167ee4893d528770","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.591Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-a5bac95fd7c6d5dc762f9d6e: start","job_status":"start","queue_duration_s":1.034702,"scheduling_latency_s":1.034718}2158gitlab # [ 403.092792] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:18.983Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"a5bac95fd7c6d5dc762f9d6e","created_at":"2026-09-25T16:01:17.579Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-7e86b06e1b2d4c83","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b7cacb52b16e8845167ee4893d528770","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.591Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-a5bac95fd7c6d5dc762f9d6e: done: 0.357343 sec","job_status":"done","queue_duration_s":1.034702,"scheduling_latency_s":1.034718,"redis_calls":564,"redis_duration_s":0.07312199999999999,"redis_read_bytes":562,"redis_write_bytes":49063,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000385,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000901,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.071836,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.322278,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.357343,"completed_at":"2026-09-25T16:01:18.983Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000602,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2159gitlab # [ 403.159972] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:18.984Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"e0bbe04efee5f97e1dbc2d1e","created_at":"2026-09-25T16:01:17.606Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-7e86b06e1b2d4c83","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"259de3219d9df0e5e28c1fd36b15575e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.617Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-e0bbe04efee5f97e1dbc2d1e: start","job_status":"start","queue_duration_s":1.366725,"scheduling_latency_s":1.366742}2160gitlab # [ 403.181519] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.002Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"e0bbe04efee5f97e1dbc2d1e","created_at":"2026-09-25T16:01:17.606Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-7e86b06e1b2d4c83","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"259de3219d9df0e5e28c1fd36b15575e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.617Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-e0bbe04efee5f97e1dbc2d1e: done: 0.018379 sec","job_status":"done","queue_duration_s":1.366725,"scheduling_latency_s":1.366742,"redis_calls":9,"redis_duration_s":0.001859,"redis_read_bytes":7,"redis_write_bytes":1119,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000399,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001073,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000387,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011273,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018379,"completed_at":"2026-09-25T16:01:19.002Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000648,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2161gitlab # [ 403.230654] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.004Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0299974b08886b5069fe5ccf","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0299974b08886b5069fe5ccf: start","job_status":"start","queue_duration_s":1.200013,"scheduling_latency_s":1.200029}2162gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/53d1e523bb688b08820df8ef6496a15b07ee0ccf/statuses2163gitlab # [ 403.244699] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.025Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0299974b08886b5069fe5ccf","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0299974b08886b5069fe5ccf: done: 0.021301 sec","job_status":"done","queue_duration_s":1.200013,"scheduling_latency_s":1.200029,"redis_calls":15,"redis_duration_s":0.002354,"redis_read_bytes":12,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000319,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000656,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001379,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013159,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021301,"completed_at":"2026-09-25T16:01:19.025Z","load_balancing_strategy":"primary","db_duration_s":0.000376,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2164gitlab # [ 403.287325] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.026Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1bc4ecc4d305d7c1e531bbda","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1bc4ecc4d305d7c1e531bbda: start","job_status":"start","queue_duration_s":1.222636,"scheduling_latency_s":1.222652}2165gitlab # [ 403.300896] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.041Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1bc4ecc4d305d7c1e531bbda","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1bc4ecc4d305d7c1e531bbda: done: 0.014909 sec","job_status":"done","queue_duration_s":1.222636,"scheduling_latency_s":1.222652,"redis_calls":13,"redis_duration_s":0.00188,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000652,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001228,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009979,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014909,"completed_at":"2026-09-25T16:01:19.041Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2166gitlab # [ 403.340865] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.042Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5bc6c7eff4cbd835985a60e3","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5bc6c7eff4cbd835985a60e3: start","job_status":"start","queue_duration_s":1.238706,"scheduling_latency_s":1.238722}2167gitlab # [ 403.353543] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.057Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5bc6c7eff4cbd835985a60e3","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5bc6c7eff4cbd835985a60e3: done: 0.014986 sec","job_status":"done","queue_duration_s":1.238706,"scheduling_latency_s":1.238722,"redis_calls":13,"redis_duration_s":0.001856,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000663,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001193,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009942,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014986,"completed_at":"2026-09-25T16:01:19.057Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2168gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/53d1e523bb688b08820df8ef6496a15b07ee0ccf/statuses, in 0.14 seconds)2169gitlab # [ 403.394398] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.058Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"238c75997774b5991fb02f1c","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-238c75997774b5991fb02f1c: start","job_status":"start","queue_duration_s":1.254808,"scheduling_latency_s":1.254824}2170gitlab # [ 403.407318] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.073Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"238c75997774b5991fb02f1c","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-238c75997774b5991fb02f1c: done: 0.014275 sec","job_status":"done","queue_duration_s":1.254808,"scheduling_latency_s":1.254824,"redis_calls":13,"redis_duration_s":0.001735,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000653,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001082,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00972,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014275,"completed_at":"2026-09-25T16:01:19.073Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2171gitlab # [ 403.447418] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.074Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b40e862ffe03cfd4371a4dbb","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b40e862ffe03cfd4371a4dbb: start","job_status":"start","queue_duration_s":1.270645,"scheduling_latency_s":1.270661}2172gitlab # [ 403.460538] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:16:01:19 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/53d1e523bb688b08820df8ef6496a15b07ee0ccf/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 842173gitlab # [ 403.463464] gitaly[914]: time="2026-09-25T16:01:19.259Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMSEB47406EVMJ2XCCZW2Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-25T16:01:49.257" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:19.258" grpc.time_ms=0.756 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2174gitlab # [ 403.476657] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.089Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b40e862ffe03cfd4371a4dbb","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b40e862ffe03cfd4371a4dbb: done: 0.014874 sec","job_status":"done","queue_duration_s":1.270645,"scheduling_latency_s":1.270661,"redis_calls":13,"redis_duration_s":0.002112,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000766,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001346,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009682,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014874,"completed_at":"2026-09-25T16:01:19.089Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2175gitlab # [ 403.517279] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.090Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6869eb9e261e80005ec0ef46","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6869eb9e261e80005ec0ef46: start","job_status":"start","queue_duration_s":1.286717,"scheduling_latency_s":1.286733}2176gitlab # [ 403.529849] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.105Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6869eb9e261e80005ec0ef46","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6869eb9e261e80005ec0ef46: done: 0.014481 sec","job_status":"done","queue_duration_s":1.286717,"scheduling_latency_s":1.286733,"redis_calls":13,"redis_duration_s":0.001833,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000639,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001194,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009724,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014481,"completed_at":"2026-09-25T16:01:19.105Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2177gitlab # [ 403.570774] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.106Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"572d3f17f3854fdf53476172","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-572d3f17f3854fdf53476172: start","job_status":"start","queue_duration_s":1.302432,"scheduling_latency_s":1.302449}2178gitlab # [ 403.585012] nix-daemon[3564]: accepted connection from pid 4612, user nixbot2179gitlab # [ 403.586002] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.122Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"572d3f17f3854fdf53476172","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-572d3f17f3854fdf53476172: done: 0.016518 sec","job_status":"done","queue_duration_s":1.302432,"scheduling_latency_s":1.302449,"redis_calls":13,"redis_duration_s":0.002058,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000716,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001342,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011651,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016518,"completed_at":"2026-09-25T16:01:19.122Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2180gitlab # [ 403.626289] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.124Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"862ae33c97096896dd66e90a","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-862ae33c97096896dd66e90a: start","job_status":"start","queue_duration_s":1.320541,"scheduling_latency_s":1.320558}2181gitlab # [ 403.639174] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.140Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"862ae33c97096896dd66e90a","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-862ae33c97096896dd66e90a: done: 0.015951 sec","job_status":"done","queue_duration_s":1.320541,"scheduling_latency_s":1.320558,"redis_calls":13,"redis_duration_s":0.002066,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00071,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001356,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010955,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015951,"completed_at":"2026-09-25T16:01:19.140Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2182gitlab # [ 403.679450] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.142Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a18c6a96ae97a3a90c6a4c88","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a18c6a96ae97a3a90c6a4c88: start","job_status":"start","queue_duration_s":1.338177,"scheduling_latency_s":1.338194}2183gitlab # [ 403.692346] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.157Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a18c6a96ae97a3a90c6a4c88","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a18c6a96ae97a3a90c6a4c88: done: 0.015488 sec","job_status":"done","queue_duration_s":1.338177,"scheduling_latency_s":1.338194,"redis_calls":13,"redis_duration_s":0.002007,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000659,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001348,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010547,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015488,"completed_at":"2026-09-25T16:01:19.157Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2184gitlab # [ 403.732628] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.158Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"79df6b4a447ae90cd8404bd5","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-79df6b4a447ae90cd8404bd5: start","job_status":"start","queue_duration_s":1.354792,"scheduling_latency_s":1.354808}2185gitlab # [ 403.745436] gitaly[914]: time="2026-09-25T16:01:19.577Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMSEMQXXMVT9H7W1Y3TJ0R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-25T16:01:49.576" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:19.576" grpc.time_ms=0.557 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2186gitlab # [ 403.758706] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.175Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"79df6b4a447ae90cd8404bd5","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-79df6b4a447ae90cd8404bd5: done: 0.016325 sec","job_status":"done","queue_duration_s":1.354792,"scheduling_latency_s":1.354808,"redis_calls":13,"redis_duration_s":0.002097,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000692,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001405,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010912,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016325,"completed_at":"2026-09-25T16:01:19.175Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2187gitlab # [ 403.801166] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.176Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"83fb6cb3e19bbaf9869a98d8","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-83fb6cb3e19bbaf9869a98d8: start","job_status":"start","queue_duration_s":1.372245,"scheduling_latency_s":1.37226}2188gitlab # [ 403.814514] gitaly[914]: time="2026-09-25T16:01:19.630Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3CMSEMQXXMVT9H7W1Y3TJ0R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-25T16:01:29.299" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-25T16:01:19.620" grpc.time_ms=9.8 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2189gitlab # [ 403.828723] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.203Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"83fb6cb3e19bbaf9869a98d8","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-83fb6cb3e19bbaf9869a98d8: done: 0.027476 sec","job_status":"done","queue_duration_s":1.372245,"scheduling_latency_s":1.37226,"redis_calls":13,"redis_duration_s":0.002921,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000832,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002089,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016074,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.027476,"completed_at":"2026-09-25T16:01:19.203Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2190gitlab # [ 403.869906] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.205Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"557ced14f4dabf15d2b1be40","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-557ced14f4dabf15d2b1be40: start","job_status":"start","queue_duration_s":1.401952,"scheduling_latency_s":1.401976}2191gitlab # [ 403.882873] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.228Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"557ced14f4dabf15d2b1be40","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-557ced14f4dabf15d2b1be40: done: 0.022772 sec","job_status":"done","queue_duration_s":1.401952,"scheduling_latency_s":1.401976,"redis_calls":13,"redis_duration_s":0.002929,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000886,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002043,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014943,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022772,"completed_at":"2026-09-25T16:01:19.228Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2192gitlab # [ 403.923034] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.230Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"70753da0bc36dbc9a5454e79","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-70753da0bc36dbc9a5454e79: start","job_status":"start","queue_duration_s":1.426339,"scheduling_latency_s":1.426362}2193gitlab # [ 403.935904] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:16:01:19 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/53d1e523bb688b08820df8ef6496a15b07ee0ccf HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 2732194gitlab # [ 403.939022] nixbot[3401]: {"ts": "2026-09-25T16:01:19.786Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/53d1e523bb688b08820df8ef6496a15b07ee0ccf \"HTTP/1.1 201 Created\""}2195gitlab # [ 403.942080] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.253Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"70753da0bc36dbc9a5454e79","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-70753da0bc36dbc9a5454e79: done: 0.023311 sec","job_status":"done","queue_duration_s":1.426339,"scheduling_latency_s":1.426362,"redis_calls":13,"redis_duration_s":0.0025589999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000877,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001682,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016036,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023311,"completed_at":"2026-09-25T16:01:19.253Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2196gitlab # [ 403.983050] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.255Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be95ea20e4e2b85e60f36492","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be95ea20e4e2b85e60f36492: start","job_status":"start","queue_duration_s":1.451085,"scheduling_latency_s":1.451103}2197gitlab # [ 403.995937] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.271Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be95ea20e4e2b85e60f36492","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be95ea20e4e2b85e60f36492: done: 0.016501 sec","job_status":"done","queue_duration_s":1.451085,"scheduling_latency_s":1.451103,"redis_calls":13,"redis_duration_s":0.002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000741,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001259,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010841,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016501,"completed_at":"2026-09-25T16:01:19.271Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2198gitlab # [ 404.036971] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.273Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"13ded0d285baf8825b3b7e7b","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-13ded0d285baf8825b3b7e7b: start","job_status":"start","queue_duration_s":1.469446,"scheduling_latency_s":1.469464}2199gitlab # [ 404.050173] gitaly[914]: time="2026-09-25T16:01:19.843Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMSEXFK6XD8GC87R9AGXMX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-25T16:01:49.841" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:19.842" grpc.time_ms=0.691 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2200gitlab # [ 404.063690] nixbot[3401]: {"ts": "2026-09-25T16:01:19.937Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/53d1e523bb688b08820df8ef6496a15b07ee0ccf \"HTTP/1.1 201 Created\""}2201gitlab # [ 404.066803] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.292Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"13ded0d285baf8825b3b7e7b","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-13ded0d285baf8825b3b7e7b: done: 0.018905 sec","job_status":"done","queue_duration_s":1.469446,"scheduling_latency_s":1.469464,"redis_calls":13,"redis_duration_s":0.0028840000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001191,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001693,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011657,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018905,"completed_at":"2026-09-25T16:01:19.292Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2202gitlab # [ 404.106982] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.294Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"36808c02b4aeab1cd838582c","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-36808c02b4aeab1cd838582c: start","job_status":"start","queue_duration_s":1.490644,"scheduling_latency_s":1.490661}2203gitlab # [ 404.120008] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:16:01:19 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/53d1e523bb688b08820df8ef6496a15b07ee0ccf HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1432204gitlab # [ 404.122594] gitaly[914]: time="2026-09-25T16:01:19.884Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M3CMSEXFK6XD8GC87R9AGXMX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-25T16:01:29.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-25T16:01:19.874" grpc.time_ms=9.459 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2205gitlab # [ 404.136449] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.313Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"36808c02b4aeab1cd838582c","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-36808c02b4aeab1cd838582c: done: 0.018764 sec","job_status":"done","queue_duration_s":1.490644,"scheduling_latency_s":1.490661,"redis_calls":13,"redis_duration_s":0.002239,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000698,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001541,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013494,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018764,"completed_at":"2026-09-25T16:01:19.313Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2206gitlab # [ 404.176838] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.315Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb8c33d9b9668a9d4da99f63","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb8c33d9b9668a9d4da99f63: start","job_status":"start","queue_duration_s":1.511514,"scheduling_latency_s":1.511537}2207gitlab # [ 404.190120] gitaly[914]: time="2026-09-25T16:01:20.010Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3CMSF2B910T1VXBKTQJ313S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-25T16:01:50.009" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-25T16:01:20.009" grpc.time_ms=0.482 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2208gitlab # [ 404.204074] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.338Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb8c33d9b9668a9d4da99f63","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb8c33d9b9668a9d4da99f63: done: 0.022888 sec","job_status":"done","queue_duration_s":1.511514,"scheduling_latency_s":1.511537,"redis_calls":13,"redis_duration_s":0.0022589999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000742,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001517,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016564,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022888,"completed_at":"2026-09-25T16:01:19.338Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2209gitlab # [ 404.245419] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.340Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5af60f989e9c418e02fa8db9","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5af60f989e9c418e02fa8db9: start","job_status":"start","queue_duration_s":1.536814,"scheduling_latency_s":1.536838}2210gitlab # [ 404.258151] gitaly[914]: time="2026-09-25T16:01:20.059Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=3 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3CMSF2B910T1VXBKTQJ313S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-25T16:01:30.010" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-25T16:01:20.050" grpc.time_ms=9.033 pid=914 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2211gitlab # [ 404.271866] gitlab-workhorse[821]: gitlab 2001:db8:1::1 - - [2026/09/25:16:01:20 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/53d1e523bb688b08820df8ef6496a15b07ee0ccf HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1682212gitlab # [ 404.274620] nixbot[3401]: {"ts": "2026-09-25T16:01:20.117Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/53d1e523bb688b08820df8ef6496a15b07ee0ccf \"HTTP/1.1 201 Created\""}2213gitlab # [ 404.277526] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.361Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5af60f989e9c418e02fa8db9","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5af60f989e9c418e02fa8db9: done: 0.02096 sec","job_status":"done","queue_duration_s":1.536814,"scheduling_latency_s":1.536838,"redis_calls":13,"redis_duration_s":0.0022979999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000779,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001519,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014609,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02096,"completed_at":"2026-09-25T16:01:19.361Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2214gitlab # [ 404.317979] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.363Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12957eb85221cdf9750d66cc","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12957eb85221cdf9750d66cc: start","job_status":"start","queue_duration_s":1.5597,"scheduling_latency_s":1.559723}2215gitlab # [ 404.330685] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.387Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12957eb85221cdf9750d66cc","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12957eb85221cdf9750d66cc: done: 0.023668 sec","job_status":"done","queue_duration_s":1.5597,"scheduling_latency_s":1.559723,"redis_calls":13,"redis_duration_s":0.0030670000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000972,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002095,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014653,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023668,"completed_at":"2026-09-25T16:01:19.387Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2216gitlab # [ 404.371431] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.389Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ba9d8c3481e5b55285b91ce9","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ba9d8c3481e5b55285b91ce9: start","job_status":"start","queue_duration_s":1.585753,"scheduling_latency_s":1.585774}2217gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/53d1e523bb688b08820df8ef6496a15b07ee0ccf/statuses2218gitlab # [ 404.385189] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.410Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ba9d8c3481e5b55285b91ce9","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ba9d8c3481e5b55285b91ce9: done: 0.020399 sec","job_status":"done","queue_duration_s":1.585753,"scheduling_latency_s":1.585774,"redis_calls":13,"redis_duration_s":0.002904,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001142,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001762,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014085,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020399,"completed_at":"2026-09-25T16:01:19.410Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2219gitlab # [ 404.426674] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.411Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc7983743d40ef6b62a72ceb","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc7983743d40ef6b62a72ceb: start","job_status":"start","queue_duration_s":1.607815,"scheduling_latency_s":1.607833}2220gitlab # [ 404.439940] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.428Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc7983743d40ef6b62a72ceb","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc7983743d40ef6b62a72ceb: done: 0.016776 sec","job_status":"done","queue_duration_s":1.607815,"scheduling_latency_s":1.607833,"redis_calls":13,"redis_duration_s":0.0025340000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000697,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001837,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011404,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016776,"completed_at":"2026-09-25T16:01:19.428Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2221gitlab # [ 404.480497] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.430Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"648d1f47132b389d2a653f68","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-648d1f47132b389d2a653f68: start","job_status":"start","queue_duration_s":1.626269,"scheduling_latency_s":1.626287}2222gitlab # [ 404.493512] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.445Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"648d1f47132b389d2a653f68","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-648d1f47132b389d2a653f68: done: 0.015745 sec","job_status":"done","queue_duration_s":1.626269,"scheduling_latency_s":1.626287,"redis_calls":13,"redis_duration_s":0.002019,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000708,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001311,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010882,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015745,"completed_at":"2026-09-25T16:01:19.445Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2223gitlab # [ 404.534174] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.447Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee1cf1ecda71b6d96915e3c9","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee1cf1ecda71b6d96915e3c9: start","job_status":"start","queue_duration_s":1.643237,"scheduling_latency_s":1.643254}2224gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/53d1e523bb688b08820df8ef6496a15b07ee0ccf/statuses, in 0.15 seconds)2225(finished: subtest: eval and build post commit statuses, in 10.41 seconds)2226(finished: run the VM test script, in 471.73 seconds)2227gitlab # [ 404.547691] gitaly[914]: time="2026-09-25T16:01:20.334Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3CMSEMQXXMVT9H7W1Y3TJ0R grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-25T16:01:20.333" grpc.time_ms=1.255 pid=914 span.kind=server system=grpc2228gitlab # [ 404.558581] gitaly[914]: time="2026-09-25T16:01:20.374Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3CMSEMQXXMVT9H7W1Y3TJ0R grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-25T16:01:20.374" grpc.time_ms=0.358 pid=914 span.kind=server system=grpc2229gitlab # [ 404.568597] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.462Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee1cf1ecda71b6d96915e3c9","created_at":"2026-09-25T16:01:17.331Z","trace_propagation_headers":{"sentry-trace":"8f171172d2a941ccbb4d56def9a7c679-b915b9b0601e4fe8","baggage":"sentry-trace_id=8f171172d2a941ccbb4d56def9a7c679,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fff3dd9a5f8e007bacd9ba6e2078056","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-25T16:01:17.803Z","job_size_bytes":2,"pid":3274,"sidekiq_tid":"m8iq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee1cf1ecda71b6d96915e3c9: done: 0.015777 sec","job_status":"done","queue_duration_s":1.643237,"scheduling_latency_s":1.643254,"redis_calls":13,"redis_duration_s":0.0018830000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000628,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001255,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010715,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015777,"completed_at":"2026-09-25T16:01:19.462Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2230test script finished in 471.79s2231cleanup2232kill QemuMachine (pid 46)2233gitlab # [ 404.614382] sidekiq-cluster[3274]: {"severity":"INFO","time":"2026-09-25T16:01:19.464Z","retry":0,"queue":"default","version":0,"store":null,"status_qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/41m77i1296n33p6liin8ynr6wh3h6b7m-python3-3.14.6/bin/python3.14)2234gitlab # e2235(finished: cleanup, in 1.70 seconds)2236additionally exposed symbols:2237 gitlab,2238 vlan1,2239 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh2240[{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-09-25T15:59:43.516Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]2241[]2242[]2243[]2244[]2245[]2246[]2247[]2248[{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '53d1e523bb688b08820df8ef6496a15b07ee0ccf', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'pending', 'eval_warnings': None, 'error': None, 'created_at': '2026-09-25T16:01:09.812057Z', 'started_at': None, 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': {'position': 1, 'blocked_by': []}}]2249[{'id': 5, 'sha': '53d1e523bb688b08820df8ef6496a15b07ee0ccf', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-25T16:01:10.083Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2250[{'id': 5, 'sha': '53d1e523bb688b08820df8ef6496a15b07ee0ccf', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-25T16:01:10.083Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2251[{'id': 5, 'sha': '53d1e523bb688b08820df8ef6496a15b07ee0ccf', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-25T16:01:10.083Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2252[{'id': 5, 'sha': '53d1e523bb688b08820df8ef6496a15b07ee0ccf', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-25T16:01:10.083Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2253[{'id': 5, 'sha': '53d1e523bb688b08820df8ef6496a15b07ee0ccf', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-25T16:01:10.083Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2254[{'id': 5, 'sha': '53d1e523bb688b08820df8ef6496a15b07ee0ccf', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-25T16:01:10.083Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2255[{'id': 5, 'sha': '53d1e523bb688b08820df8ef6496a15b07ee0ccf', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-25T16:01:10.083Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2256[{'id': 5, 'sha': '53d1e523bb688b08820df8ef6496a15b07ee0ccf', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-25T16:01:10.083Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2257[{'id': 5, 'sha': '53d1e523bb688b08820df8ef6496a15b07ee0ccf', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-25T16:01:10.083Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2258[{'id': 5, 'sha': '53d1e523bb688b08820df8ef6496a15b07ee0ccf', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 9s (1 warning)', 'created_at': '2026-09-25T16:01:10.083Z', 'started_at': None, 'finished_at': '2026-09-25T16:01:19.658Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': '53d1e523bb688b08820df8ef6496a15b07ee0ccf', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '0 attributes built', 'created_at': '2026-09-25T16:01:19.907Z', 'started_at': None, 'finished_at': '2026-09-25T16:01:20.085Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]