vm-test-run-nixbot-gitlab
checks.aarch64-linux.nixbot-gitlab
· build #396
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 3600 seconds6run the VM test script7start all VMs8gitlab: starting vm9gitlab # Disk image does not exist, creating the virtualisation disk image...10gitlab: QEMU running (pid 46)11gitlab # Formatting '/build/vm-state-gitlab/tmp.3SgfpkH82P', fmt=raw size=107374182412gitlab # mke2fs 1.47.4 (6-Mar-2025)13gitlab # Discarding device blocks: 0/262144 done14gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes15gitlab # Filesystem UUID: d82bc7c1-8ff8-443e-b559-388d38f5de9f16(finished: start all VMs, in 74.62 seconds)17gitlab # Superblock backups stored on blocks:18gitlab: waiting for unit setup-gitlab.service19gitlab # 32768, 98304, 163840, 22937620gitlab: waiting for the VM to finish booting21gitlab # 22gitlab # Allocating group tables: 0/8 done23gitlab # Writing inode tables: 0/8 done24gitlab # Creating journal (8192 blocks): done25gitlab # Writing superblocks and filesystem accounting information: 0/8 done26gitlab # 27gitlab # Virtualisation disk image created.28gitlab # Creating Nix store image...29gitlab # Created Nix store image.30gitlab # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]31gitlab # [ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Thu Aug 6 17:27:09 UTC 202632gitlab # [ 0.000000] KASLR enabled33gitlab # [ 0.000000] random: crng init done34gitlab # [ 0.000000] Machine model: linux,dummy-virt35gitlab # [ 0.000000] efi: UEFI not found.36gitlab # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT37gitlab # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x00000001bfffffff]38gitlab # [ 0.000000] NODE_DATA(0) allocated [mem 0x1bfdeb740-0x1bfdeeebf]39gitlab # [ 0.000000] Zone ranges:40gitlab # [ 0.000000] DMA [mem 0x0000000040000000-0x00000000ffffffff]41gitlab # [ 0.000000] DMA32 empty42gitlab # [ 0.000000] Normal [mem 0x0000000100000000-0x00000001bfffffff]43gitlab # [ 0.000000] Device empty44gitlab # [ 0.000000] Movable zone start for each node45gitlab # [ 0.000000] Early memory node ranges46gitlab # [ 0.000000] node 0: [mem 0x0000000040000000-0x00000001bfffffff]47gitlab # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000001bfffffff]48gitlab # [ 0.000000] cma: Reserved 32 MiB at 0x00000000fe00000049gitlab # [ 0.000000] psci: probing for conduit method from DT.50gitlab # [ 0.000000] psci: PSCIv1.3 detected in firmware.51gitlab # [ 0.000000] psci: Using standard PSCI v0.2 function IDs52gitlab # [ 0.000000] psci: Trusted OS migration not required53gitlab # [ 0.000000] psci: SMC Calling Convention v1.154gitlab # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)55gitlab # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u31129656gitlab # [ 0.000000] Detected PIPT I-cache on CPU057gitlab # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)58gitlab # [ 0.000000] CPU features: detected: GICv3 CPU interface59gitlab # [ 0.000000] CPU features: detected: Spectre-v460gitlab # [ 0.000000] CPU features: detected: Spectre-BHB61gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_3862gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_2363gitlab # [ 0.000000] alternatives: applying boot alternatives64gitlab # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/8sxiiab1vflpgf1s6qwyprgzkhnnrwlc-nixos-system-gitlab-test/init regInfo=/nix/store/z87rh5qxhfq1sccd66543ivqd5dyja1c-closure-info/registration console=ttyAMA0,115200n8 console=tty065gitlab # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/z87rh5qxhfq1sccd66543ivqd5dyja1c-closure-info/registration", will be passed to user space.66gitlab # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes67gitlab # [ 0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)68gitlab # [ 0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)69gitlab # [ 0.000000] software IO TLB: area num 4.70gitlab # [ 0.000000] software IO TLB: mapped [mem 0x00000000fa000000-0x00000000fe000000] (64MB)71gitlab # [ 0.000000] Fallback order for Node 0: 072gitlab # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 157286473gitlab # [ 0.000000] Policy zone: Normal74gitlab # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off75gitlab # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=176gitlab # [ 0.000000] allocated 12582912 bytes of page_ext77gitlab # [ 0.000000] ftrace: allocating 74867 entries in 294 pages78gitlab # [ 0.000000] ftrace: allocated 294 pages with 4 groups79gitlab # [ 0.000000] rcu: Hierarchical RCU implementation.80gitlab # [ 0.000000] rcu: RCU event tracing is enabled.81gitlab # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.82gitlab # [ 0.000000] Trampoline variant of Tasks RCU enabled.83gitlab # [ 0.000000] Rude variant of Tasks RCU enabled.84gitlab # [ 0.000000] Tracing variant of Tasks RCU enabled.85gitlab # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.86gitlab # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=487gitlab # [ 0.000000] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.88gitlab # [ 0.000000] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.89gitlab # [ 0.000000] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.90gitlab # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 091gitlab # [ 0.000000] GICv3: 256 SPIs implemented92gitlab # [ 0.000000] GICv3: 0 Extended SPIs implemented93gitlab # [ 0.000000] Root IRQ handler: gic_handle_irq94gitlab # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI95gitlab # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=096gitlab # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a000097gitlab # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]98gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @100f10000 (indirect, esz 8, psz 64K, shr 1)99gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @100f20000 (flat, esz 8, psz 64K, shr 1)100gitlab # [ 0.000000] GICv3: using LPI property table @0x0000000100f30000101gitlab # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000100f40000102gitlab # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.103gitlab # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns104gitlab # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).105gitlab # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns106gitlab # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns107gitlab # [ 0.000029] arm-pv: using stolen time PV108gitlab # [ 0.000381] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)109gitlab # [ 0.000615] Console: colour dummy device 80x25110gitlab # [ 0.000623] printk: legacy console [tty0] enabled111gitlab # [ 0.000803] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)112gitlab # [ 0.000810] pid_max: default: 32768 minimum: 301113gitlab # [ 0.000892] LSM: initializing lsm=capability,landlock,yama,bpf,ima114gitlab # [ 0.001009] landlock: Up and running.115gitlab # [ 0.001012] Yama: becoming mindful.116gitlab # [ 0.001414] LSM support for eBPF active117gitlab # [ 0.001631] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)118gitlab # [ 0.001748] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)119gitlab # [ 0.002927] cacheinfo: Unable to detect cache hierarchy for CPU 0120gitlab # [ 0.003641] rcu: Hierarchical SRCU implementation.121gitlab # [ 0.003645] rcu: Max phase no-delay instances is 1000.122gitlab # [ 0.003796] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level123gitlab # [ 0.004921] fsl-mc MSI: its@8080000 domain created124gitlab # [ 0.005017] EFI services will not be available.125gitlab # [ 0.005235] smp: Bringing up secondary CPUs ...126gitlab # [ 0.005862] Detected PIPT I-cache on CPU1127gitlab # [ 0.005931] GICv3: CPU1: found redistributor 1 region 0:0x00000000080c0000128gitlab # [ 0.006004] GICv3: CPU1: using allocated LPI pending table @0x0000000100f50000129gitlab # [ 0.006079] CPU1: Booted secondary processor 0x0000000001 [0xc00fac40]130gitlab # [ 0.007147] Detected PIPT I-cache on CPU2131gitlab # [ 0.007222] GICv3: CPU2: found redistributor 2 region 0:0x00000000080e0000132gitlab # [ 0.007295] GICv3: CPU2: using allocated LPI pending table @0x0000000100f60000133gitlab # [ 0.007371] CPU2: Booted secondary processor 0x0000000002 [0xc00fac40]134gitlab # [ 0.008194] Detected PIPT I-cache on CPU3135gitlab # [ 0.008268] GICv3: CPU3: found redistributor 3 region 0:0x0000000008100000136gitlab # [ 0.008342] GICv3: CPU3: using allocated LPI pending table @0x0000000100f70000137gitlab # [ 0.008417] CPU3: Booted secondary processor 0x0000000003 [0xc00fac40]138gitlab # [ 0.008829] smp: Brought up 1 node, 4 CPUs139gitlab # [ 0.008877] SMP: Total of 4 processors activated.140gitlab # [ 0.008882] CPU: All CPU(s) started at EL1141gitlab # [ 0.008897] CPU features: detected: Branch Target Identification142gitlab # [ 0.008901] CPU features: detected: ARMv8.4 Translation Table Level143gitlab # [ 0.008904] CPU features: detected: Instruction cache invalidation not required for I/D coherence144gitlab # [ 0.008908] CPU features: detected: Data cache clean to the PoU not required for I/D coherence145gitlab # [ 0.008911] CPU features: detected: Common not Private translations146gitlab # [ 0.008915] CPU features: detected: CRC32 instructions147gitlab # [ 0.008917] CPU features: detected: Data cache clean to Point of Deep Persistence148gitlab # [ 0.008921] CPU features: detected: Data cache clean to Point of Persistence149gitlab # [ 0.008924] CPU features: detected: Data independent timing control (DIT)150gitlab # [ 0.008927] CPU features: detected: E0PD151gitlab # [ 0.008929] CPU features: detected: Enhanced Counter Virtualization152gitlab # [ 0.008932] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)153gitlab # [ 0.008936] CPU features: detected: Enhanced Virtualization Traps154gitlab # [ 0.008939] CPU features: detected: Fine Grained Traps155gitlab # [ 0.008942] CPU features: detected: Generic authentication (architected QARMA5 algorithm)156gitlab # [ 0.008946] CPU features: detected: RCpc load-acquire (LDAPR)157gitlab # [ 0.008949] CPU features: detected: LSE atomic instructions158gitlab # [ 0.008952] CPU features: detected: Privileged Access Never159gitlab # [ 0.008954] CPU features: detected: PMUv3160gitlab # [ 0.008957] CPU features: detected: RAS Extension Support161gitlab # [ 0.008960] CPU features: detected: RASv1p1 Extension Support162gitlab # [ 0.008963] CPU features: detected: Random Number Generator163gitlab # [ 0.008965] CPU features: detected: Speculation barrier (SB)164gitlab # [ 0.008968] CPU features: detected: Stage-2 Force Write-Back165gitlab # [ 0.008971] CPU features: detected: TLB range maintenance instructions166gitlab # [ 0.008974] CPU features: detected: Speculative Store Bypassing Safe (SSBS)167gitlab # [ 0.009074] alternatives: applying system-wide alternatives168gitlab # [ 0.012467] CPU features: detected: BBM Level 2 without TLB conflict abort169gitlab # [ 0.012833] Memory: 5963748K/6291456K available (24448K kernel code, 7090K rwdata, 26340K rodata, 4736K init, 1106K bss, 272908K reserved, 32768K cma-reserved)170gitlab # [ 0.014064] devtmpfs: initialized171gitlab # [ 0.018578] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)172gitlab # [ 0.018658] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).173gitlab # [ 0.018875] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL174gitlab # [ 0.018880] 0 pages in range for non-PLT usage175gitlab # [ 0.018881] 508288 pages in range for PLT usage176gitlab # [ 0.019044] pinctrl core: initialized pinctrl subsystem177gitlab # [ 0.020134] DMI not present or invalid.178gitlab # [ 0.024083] NET: Registered PF_NETLINK/PF_ROUTE protocol family179gitlab # [ 0.026910] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations180gitlab # [ 0.027352] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations181gitlab # [ 0.029701] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations182gitlab # [ 0.029730] audit: initializing netlink subsys (disabled)183gitlab # [ 0.030089] audit: type=2000 audit(0.028:1): state=initialized audit_enabled=0 res=1184gitlab # [ 0.030658] thermal_sys: Registered thermal governor 'fair_share'185gitlab # [ 0.030660] thermal_sys: Registered thermal governor 'bang_bang'186gitlab # [ 0.030665] thermal_sys: Registered thermal governor 'step_wise'187gitlab # [ 0.030667] thermal_sys: Registered thermal governor 'user_space'188gitlab # [ 0.030670] thermal_sys: Registered thermal governor 'power_allocator'189gitlab # [ 0.030740] cpuidle: using governor ladder190gitlab # [ 0.030761] cpuidle: using governor menu191gitlab # [ 0.030970] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.192gitlab # [ 0.031116] ASID allocator initialised with 65536 entries193gitlab # [ 0.032477] Serial: AMBA PL011 UART driver194gitlab # [ 0.038385] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1195gitlab # [ 0.038625] printk: console [ttyAMA0] enabled196gitlab # [ 0.044646] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages197gitlab # [ 0.044658] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page198gitlab # [ 0.044663] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages199gitlab # [ 0.044668] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page200gitlab # [ 0.044672] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages201gitlab # [ 0.044676] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page202gitlab # [ 0.044681] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages203gitlab # [ 0.044685] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page204gitlab # [ 0.051546] fbcon: Taking over console205gitlab # [ 0.051559] ACPI: Interpreter disabled.206gitlab # [ 0.052865] iommu: Default domain type: Translated207gitlab # [ 0.052875] iommu: DMA domain TLB invalidation policy: strict mode208gitlab # [ 0.068381] SCSI subsystem initialized209gitlab # [ 0.068988] usbcore: registered new interface driver usbfs210gitlab # [ 0.069039] usbcore: registered new interface driver hub211gitlab # [ 0.069087] usbcore: registered new device driver usb212gitlab # [ 0.069560] pps_core: LinuxPPS API ver. 1 registered213gitlab # [ 0.069566] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>214gitlab # [ 0.069579] PTP clock support registered215gitlab # [ 0.069634] EDAC MC: Ver: 3.0.0216gitlab # [ 0.070011] scmi_core: SCMI protocol bus registered217gitlab # [ 0.070717] FPGA manager framework218gitlab # [ 0.071899] vgaarb: loaded219gitlab # [ 0.073310] clocksource: Switched to clocksource arch_sys_counter220gitlab # [ 0.074103] VFS: Disk quotas dquot_6.6.0221gitlab # [ 0.074139] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)222gitlab # [ 0.074480] netfs: FS-Cache loaded223gitlab # [ 0.074627] pnp: PnP ACPI: disabled224gitlab # [ 0.079514] NET: Registered PF_INET protocol family225gitlab # [ 0.080566] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)226gitlab # [ 0.109042] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)227gitlab # [ 0.109094] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)228gitlab # [ 0.109137] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)229gitlab # [ 0.109443] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)230gitlab # [ 0.109998] TCP: Hash tables configured (established 65536 bind 65536)231gitlab # [ 0.110098] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)232gitlab # [ 0.110161] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)233gitlab # [ 0.110275] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)234gitlab # [ 0.110477] NET: Registered PF_UNIX/PF_LOCAL protocol family235gitlab # [ 0.110496] NET: Registered PF_XDP protocol family236gitlab # [ 0.110510] PCI: CLS 0 bytes, default 64237gitlab # [ 0.110743] Trying to unpack rootfs image as initramfs...238gitlab # [ 0.122153] kvm [1]: HYP mode not available239gitlab # [ 0.167831] Initialise system trusted keyrings240gitlab # [ 0.168057] workingset: timestamp_bits=42 max_order=21 bucket_order=0241gitlab # [ 0.168651] squashfs: version 4.0 (2009/01/31) Phillip Lougher242gitlab # [ 0.168722] 9p: Installing v9fs 9p2000 file system support243gitlab # [ 0.180684] Key type asymmetric registered244gitlab # [ 0.180689] Asymmetric key parser 'x509' registered245gitlab # [ 0.180748] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)246gitlab # [ 0.180921] io scheduler mq-deadline registered247gitlab # [ 0.180928] io scheduler kyber registered248gitlab # [ 0.190734] pl061_gpio 9030000.pl061: PL061 GPIO chip registered249gitlab # [ 0.194203] ledtrig-cpu: registered to indicate activity on CPUs250gitlab # [ 0.194727] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:251gitlab # [ 0.194742] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000252gitlab # [ 0.194754] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000253gitlab # [ 0.194759] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000254gitlab # [ 0.194781] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits255gitlab # [ 0.194805] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]256gitlab # [ 0.194891] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00257gitlab # [ 0.194899] pci_bus 0000:00: root bus resource [bus 00-ff]258gitlab # [ 0.194903] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]259gitlab # [ 0.194906] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]260gitlab # [ 0.194909] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]261gitlab # [ 0.195018] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint262gitlab # [ 0.195584] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint263gitlab # [ 0.195819] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]264gitlab # [ 0.195833] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]265gitlab # [ 0.195863] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]266gitlab # [ 0.195876] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]267gitlab # [ 0.196485] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint268gitlab # [ 0.196761] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]269gitlab # [ 0.196773] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]270gitlab # [ 0.196802] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]271gitlab # [ 0.197423] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint272gitlab # [ 0.197694] pci 0000:00:03.0: BAR 0 [io 0x0000-0x001f]273gitlab # [ 0.197708] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]274gitlab # [ 0.197738] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]275gitlab # [ 0.198371] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint276gitlab # [ 0.198584] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]277gitlab # [ 0.198597] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]278gitlab # [ 0.198632] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]279gitlab # [ 0.199245] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint280gitlab # [ 0.199485] pci 0000:00:05.0: BAR 0 [io 0x0000-0x007f]281gitlab # [ 0.199498] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]282gitlab # [ 0.199528] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]283gitlab # [ 0.200149] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint284gitlab # [ 0.200388] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]285gitlab # [ 0.200401] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]286gitlab # [ 0.200434] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]287gitlab # [ 0.201079] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint288gitlab # [ 0.201301] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]289gitlab # [ 0.201327] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]290gitlab # [ 0.201369] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]291gitlab # [ 0.201382] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]292gitlab # [ 0.202080] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint293gitlab # [ 0.202367] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]294gitlab # [ 0.202404] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]295gitlab # [ 0.203082] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint296gitlab # [ 0.203359] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]297gitlab # [ 0.203388] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]298gitlab # [ 0.203931] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint299gitlab # [ 0.204192] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]300gitlab # [ 0.204548] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint301gitlab # [ 0.205009] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]302gitlab # [ 0.205023] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]303gitlab # [ 0.205053] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]304gitlab # [ 0.205674] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint305gitlab # [ 0.205904] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]306gitlab # [ 0.205917] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]307gitlab # [ 0.205958] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]308gitlab # [ 0.206743] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned309gitlab # [ 0.206749] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned310gitlab # [ 0.206753] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned311gitlab # [ 0.206803] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned312gitlab # [ 0.206864] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned313gitlab # [ 0.206922] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned314gitlab # [ 0.206986] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned315gitlab # [ 0.207046] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned316gitlab # [ 0.207099] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned317gitlab # [ 0.207156] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned318gitlab # [ 0.207218] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned319gitlab # [ 0.207269] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned320gitlab # [ 0.207360] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned321gitlab # [ 0.207416] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned322gitlab # [ 0.207437] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned323gitlab # [ 0.207457] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned324gitlab # [ 0.207477] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned325gitlab # [ 0.207497] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned326gitlab # [ 0.207519] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned327gitlab # [ 0.207549] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned328gitlab # [ 0.207570] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned329gitlab # [ 0.207592] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned330gitlab # [ 0.207613] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned331gitlab # [ 0.207635] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned332gitlab # [ 0.207657] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned333gitlab # [ 0.207677] pci 0000:00:05.0: BAR 0 [io 0x1000-0x107f]: assigned334gitlab # [ 0.207696] pci 0000:00:06.0: BAR 0 [io 0x1080-0x10ff]: assigned335gitlab # [ 0.207716] pci 0000:00:0b.0: BAR 0 [io 0x1100-0x113f]: assigned336gitlab # [ 0.207736] pci 0000:00:01.0: BAR 0 [io 0x1140-0x115f]: assigned337gitlab # [ 0.207761] pci 0000:00:02.0: BAR 0 [io 0x1160-0x117f]: assigned338gitlab # [ 0.207782] pci 0000:00:03.0: BAR 0 [io 0x1180-0x119f]: assigned339gitlab # [ 0.207803] pci 0000:00:04.0: BAR 0 [io 0x11a0-0x11bf]: assigned340gitlab # [ 0.207828] pci 0000:00:07.0: BAR 0 [io 0x11c0-0x11df]: assigned341gitlab # [ 0.207848] pci 0000:00:0c.0: BAR 0 [io 0x11e0-0x11ff]: assigned342gitlab # [ 0.207878] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]343gitlab # [ 0.207883] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]344gitlab # [ 0.207886] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]345gitlab # [ 0.209343] pci 0000:00:0a.0: enabling device (0000 -> 0002)346gitlab # [ 0.222081] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)347gitlab # [ 0.224579] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)348gitlab # [ 0.226771] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)349gitlab # [ 0.229067] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)350gitlab # [ 0.231344] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)351gitlab # [ 0.235800] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)352gitlab # [ 0.238263] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)353gitlab # [ 0.240426] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)354gitlab # [ 0.242664] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)355gitlab # [ 0.244690] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)356gitlab # [ 0.248657] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)357gitlab # [ 0.253933] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled358gitlab # [ 0.255603] msm_serial: driver initialized359gitlab # [ 0.255738] SuperH (H)SCI(F) driver initialized360gitlab # [ 0.255787] STM32 USART driver initialized361gitlab # [ 0.281670] loop: module loaded362gitlab # [ 0.281970] virtio_blk virtio4: 4/0/0 default/read/poll queues363gitlab # [ 0.284294] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)364gitlab # [ 0.286572] virtio_blk virtio5: 4/0/0 default/read/poll queues365gitlab # [ 0.288738] virtio_blk virtio5: [vdb] 12285776 512-byte logical blocks (6.29 GB/5.86 GiB)366gitlab # [ 0.291305] megasas: 07.734.00.00-rc1367gitlab # [ 0.292220] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]368gitlab # [ 0.295008] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000369gitlab # [ 0.295037] Intel/Sharp Extended Query Table at 0x0031370gitlab # [ 0.296784] Using buffer write method371gitlab # [ 0.296836] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]372gitlab # [ 0.298712] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000373gitlab # [ 0.298738] Intel/Sharp Extended Query Table at 0x0031374gitlab # [ 0.300453] Using buffer write method375gitlab # [ 0.300476] Concatenating MTD devices:376gitlab # [ 0.300478] (0): "0.flash"377gitlab # [ 0.300480] (1): "0.flash"378gitlab # [ 0.300482] into device "0.flash"379gitlab # [ 0.569518] Freeing initrd memory: 27184K380gitlab # [ 0.579288] tun: Universal TUN/TAP device driver, 1.6381gitlab # [ 0.584415] thunder_xcv, ver 1.0382gitlab # [ 0.584481] thunder_bgx, ver 1.0383gitlab # [ 0.584527] nicpf, ver 1.0384gitlab # [ 0.585493] e1000: Intel(R) PRO/1000 Network Driver385gitlab # [ 0.585500] e1000: Copyright (c) 1999-2006 Intel Corporation.386gitlab # [ 0.585549] e1000e: Intel(R) PRO/1000 Network Driver387gitlab # [ 0.585556] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.388gitlab # [ 0.585594] igb: Intel(R) Gigabit Ethernet Network Driver389gitlab # [ 0.585600] igb: Copyright (c) 2007-2014 Intel Corporation.390gitlab # [ 0.585646] igbvf: Intel(R) Gigabit Virtual Function Network Driver391gitlab # [ 0.585652] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.392gitlab # [ 0.585869] sky2: driver version 1.30393gitlab # [ 0.588437] usbcore: registered new interface driver usb-storage394gitlab # [ 0.588510] usbcore: registered new interface driver usbserial_generic395gitlab # [ 0.588531] usbserial: USB Serial support registered for generic396gitlab # [ 0.589008] ehci-pci 0000:00:0a.0: EHCI Host Controller397gitlab # [ 0.589066] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 1398gitlab # [ 0.589612] hv_vmbus: registering driver hyperv_keyboard399gitlab # [ 0.589665] ehci-pci 0000:00:0a.0: irq 15, io mem 0x10089000400gitlab # [ 0.590998] rtc-pl031 9010000.pl031: registered as rtc0401gitlab # [ 0.591034] rtc-pl031 9010000.pl031: setting system clock to 2026-10-02T14:09:18 UTC (1790950158)402gitlab # [ 0.591534] i2c_dev: i2c /dev entries driver403gitlab # [ 0.595930] sdhci: Secure Digital Host Controller Interface driver404gitlab # [ 0.595938] sdhci: Copyright(c) Pierre Ossman405gitlab # [ 0.596367] Synopsys Designware Multimedia Card Interface Driver406gitlab # [ 0.597024] sdhci-pltfm: SDHCI platform and OF driver helper407gitlab # [ 0.599676] hid: raw HID events driver (C) Jiri Kosina408gitlab # [ 0.600134] usbcore: registered new interface driver usbhid409gitlab # [ 0.600140] usbhid: USB HID core driver410gitlab # [ 0.601373] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.00411gitlab # [ 0.602525] hub 1-0:1.0: USB hub found412gitlab # [ 0.602592] hub 1-0:1.0: 6 ports detected413gitlab # [ 0.606846] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available414gitlab # [ 0.609250] drop_monitor: Initializing network drop monitor service415gitlab # [ 0.609445] NET: Registered PF_INET6 protocol family416gitlab # [ 0.610837] Segment Routing with IPv6417gitlab # [ 0.610864] In-situ OAM (IOAM) with IPv6418gitlab # [ 0.610957] NET: Registered PF_PACKET protocol family419gitlab # [ 0.611145] 9pnet: Installing 9P2000 support420gitlab # [ 0.613888] Key type dns_resolver registered421gitlab # [ 0.622985] registered taskstats version 1422gitlab # [ 0.623248] Loading compiled-in X.509 certificates423gitlab # [ 0.634553] Demotion targets for Node 0: null424gitlab # [ 0.634766] Key type .fscrypt registered425gitlab # [ 0.634772] Key type fscrypt-provisioning registered426gitlab # [ 0.634925] ima: No TPM chip found, activating TPM-bypass!427gitlab # [ 0.634946] ima: Allocated hash algorithm: sha1428gitlab # [ 0.634971] ima: No architecture policies found429gitlab # [ 0.636003] input: gpio-keys as /devices/platform/gpio-keys/input/input0430gitlab # [ 0.658372] clk: Disabling unused clocks431gitlab # [ 0.658405] PM: genpd: Disabling unused power domains432gitlab # [ 0.662384] Freeing unused kernel memory: 4736K433gitlab # [ 0.662563] Run /init as init process434gitlab # [ 0.692798] systemd[1]: Successfully made /usr/ read-only.435gitlab # [ 0.849377] usb 1-1: new high-speed USB device number 2 using ehci-pci436gitlab # [ 0.997456] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1437gitlab # [ 1.027015] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)438gitlab # [ 1.027054] systemd[1]: Detected virtualization qemu.439gitlab # [ 1.027091] systemd[1]: Detected architecture arm64.440gitlab # [ 1.027099] systemd[1]: Running in initrd.441gitlab # [ 1.027551] systemd[1]: Initializing machine ID from random generator.442gitlab # [ 1.027764] systemd[1]: Hostname set to <gitlab>.443gitlab # [ 1.105759] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input0444gitlab # [ 1.225355] usb 1-2: new high-speed USB device number 3 using ehci-pci445gitlab # [ 1.235698] systemd[1]: bpf-restrict-fs: LSM BPF program attached446gitlab # [ 1.304144] systemd[1]: Queued start job for default target Initrd Default Target.447gitlab # [ 1.344531] systemd[1]: Created slice Slice /system/modprobe.448gitlab # [ 1.344866] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.449gitlab # [ 1.344907] systemd[1]: Expecting device /dev/disk/by-label/nix-store...450gitlab # [ 1.344920] systemd[1]: Expecting device /dev/disk/by-label/nixos...451gitlab # [ 1.344946] systemd[1]: Reached target Path Units.452gitlab # [ 1.344964] systemd[1]: Reached target Slice Units.453gitlab # [ 1.344983] systemd[1]: Reached target Swaps.454gitlab # [ 1.345002] systemd[1]: Reached target Timer Units.455gitlab # [ 1.345199] systemd[1]: Listening on D-Bus System Message Bus Socket.456gitlab # [ 1.345565] systemd[1]: Listening on Journal Socket (/dev/log).457gitlab # [ 1.345736] systemd[1]: Listening on Journal Sockets.458gitlab # [ 1.345894] systemd[1]: Listening on udev Control Socket.459gitlab # [ 1.345988] systemd[1]: Listening on udev Kernel Socket.460gitlab # [ 1.346010] systemd[1]: Reached target Socket Units.461gitlab # [ 1.347856] systemd[1]: Starting Create List of Static Device Nodes...462gitlab # [ 1.349869] systemd[1]: Starting Load Kernel Module 9pnet_virtio...463gitlab # [ 1.349960] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs464gitlab # [ 1.351886] systemd[1]: Mounting Kernel Configuration File System...465gitlab # [ 1.368503] systemd[1]: Starting Journal Service...466gitlab # [ 1.370975] systemd[1]: Starting Load Kernel Modules...467gitlab # [ 1.371133] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os468gitlab # [ 1.376089] systemd[1]: Starting Coldplug All udev Devices...469gitlab # [ 1.378627] systemd[1]: Finished Create List of Static Device Nodes.470gitlab # [ 1.380342] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.471gitlab # [ 1.380855] systemd[1]: Finished Load Kernel Module 9pnet_virtio.472gitlab # [ 1.381387] systemd[1]: Mounted Kernel Configuration File System.473gitlab # [ 1.383964] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...474gitlab # [ 1.401470] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2475gitlab # [ 1.402243] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input0476gitlab # [ 1.405283] systemd-journald[96]: Collecting audit messages is disabled.477gitlab # [ 1.409539] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.478gitlab # [ 1.411483] systemd[1]: Starting Create Static Device Nodes in /dev...479gitlab # [ 1.416497] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.480gitlab # [ 1.421380] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev481gitlab # [ 1.423522] [drm] pci: virtio-gpu-pci detected at 0000:00:08.0482gitlab # [ 1.423725] [drm] features: -virgl +edid -resource_blob -host_visible483gitlab # [ 1.423741] [drm] features: -context_init484gitlab # [ 1.424907] [drm] number of scanouts: 1485gitlab # [ 1.424936] [drm] number of cap sets: 0486gitlab # [ 1.430542] systemd[1]: Finished Create Static Device Nodes in /dev.487gitlab # [ 1.430720] systemd[1]: Reached target Preparation for Local File Systems.488gitlab # [ 1.430741] systemd[1]: Reached target Local File Systems.489gitlab # [ 1.434120] systemd[1]: Starting Rule-based Manager for Device Events and Files...490gitlab # [ 1.437987] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic491gitlab # [ 1.438005] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 0492gitlab # [ 1.449854] Console: switching to colour frame buffer device 160x50493gitlab # [ 1.455916] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device494gitlab # [ 1.471091] systemd[1]: Finished Load Kernel Modules.495gitlab # [ 1.471335] systemd[1]: Started Journal Service.496gitlab # [ 1.470155] systemd-modules-load[97]: Using 4 probe threads497gitlab # [ 1.473673] systemd-modules-load[97]: Module 'virtio_balloon' is built in498gitlab # [ 1.474822] systemd-modules-load[97]: Module 'virtio_console' is built in499gitlab # [ 1.475975] systemd-modules-load[97]: Module 'virtio_rng' is built in500gitlab # [ 1.477127] systemd-modules-load[97]: Inserted module 'dm_mod'501gitlab # [ 1.478421] systemd-udevd[105]: Using default interface naming scheme 'v261'.502gitlab # [ 1.479721] systemd-modules-load[97]: Inserted module 'virtio_gpu'503gitlab # [ 1.481214] systemd[1]: Starting Apply Kernel Variables...504gitlab # [ 1.482572] systemd[1]: Starting Create System Files and Directories...505gitlab # [ 1.492812] systemd[1]: Started Rule-based Manager for Device Events and Files.506gitlab # [ 1.495589] systemd[1]: Finished Apply Kernel Variables.507gitlab # [ 1.498556] systemd[1]: Finished Create System Files and Directories.508gitlab # [ 1.520456] systemd[1]: Starting Virtual Console Setup...509gitlab # [ 1.557703] systemd-vconsole-setup[130]: Configuration of first virtual console was skipped, ignoring remaining ones.510gitlab # [ 1.561790] systemd[1]: Finished Virtual Console Setup.511gitlab # [ 2.010383] systemd[1]: Finished Coldplug All udev Devices.512gitlab # [ 2.011741] systemd[1]: Reached target System Initialization.513gitlab # [ 2.013162] systemd[1]: Reached target Basic System.514gitlab # [ 2.145924] (udev-worker)[119]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.515gitlab # [ 2.149290] (udev-worker)[119]: Network interface NamePolicy= disabled on kernel command line.516gitlab # [ 2.176092] systemd[1]: Found device /dev/disk/by-label/nixos.517gitlab # [ 2.182430] (udev-worker)[122]: Network interface NamePolicy= disabled on kernel command line.518gitlab # [ 2.190628] systemd[1]: Reached target Initrd Root Device.519gitlab # [ 2.192810] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...520gitlab # [ 2.205703] systemd[1]: Found device /dev/disk/by-label/nix-store.521gitlab # [ 2.232212] systemd-fsck[152]: nixos: clean, 12/65536 files, 13019/262144 blocks522gitlab # [ 2.238738] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.523gitlab # [ 2.377819] systemd[1]: Mounting /sysroot...524gitlab # [ 2.422378] EXT4-fs (vda): mounted filesystem d82bc7c1-8ff8-443e-b559-388d38f5de9f r/w with ordered data mode. Quota mode: none.525gitlab # [ 2.424276] systemd[1]: Mounted /sysroot.526gitlab # [ 2.425341] systemd[1]: Reached target Initrd Root File System.527gitlab # [ 2.429183] systemd[1]: Mounting /sysroot/nix/.ro-store...528gitlab # [ 2.456097] systemd[1]: Mounting /sysroot/nix/.rw-store...529gitlab # [ 2.461353] systemd[1]: Mounting /sysroot/run...530gitlab # [ 2.469308] systemd[1]: Mounting /sysroot/tmp/shared...531gitlab # [ 2.488167] systemd[1]: Mounting /sysroot/tmp/xchg...532gitlab # [ 2.500372] systemd[1]: Starting Mountpoints Configured in the Real Root...533gitlab # [ 2.519165] systemd-sysroot-fstab-check[166]: /sysroot should be mounted in the initrd, will request daemon-reload.534gitlab # [ 2.527638] erofs (device vdb): mounted with root inode @ nid 128.535gitlab # [ 2.553777] systemd[1]: Mounted /sysroot/nix/.ro-store.536gitlab # [ 2.556373] systemd[1]: Mounted /sysroot/nix/.rw-store.537gitlab # [ 2.558479] systemd[1]: Mounted /sysroot/run.538gitlab # [ 2.581212] systemd[1]: Mounted /sysroot/tmp/shared.539gitlab # [ 2.587898] systemd[1]: Mounted /sysroot/tmp/xchg.540gitlab # [ 2.592207] systemd[1]: Reload requested from client PID 166 ('systemd-sysroot') (unit initrd-parse-etc.service)...541gitlab # [ 2.593804] systemd[1]: Reloading...542gitlab # [ 2.691530] systemd[1]: Reloading finished in 101 ms.543gitlab # [ 2.713589] (udev-worker)[124]: mtd0ro: Failed to find and pin callout binary "/nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/mtd_probe": No such file or directory544gitlab # [ 2.716220] (udev-worker)[124]: mtd0ro: /nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory545gitlab # [ 2.768710] systemd-sysroot-fstab-check[166]: Requesting initrd-fs.target/start/replace...546gitlab # [ 2.774909] systemd-sysroot-fstab-check[166]: Requesting swap.target/start/replace...547gitlab # [ 2.780373] systemd[1]: initrd-parse-etc.service: Deactivated successfully.548gitlab # [ 2.781976] systemd[1]: Finished Mountpoints Configured in the Real Root.549gitlab # [ 2.783494] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.550gitlab # [ 2.790945] systemd[1]: Starting rw-sysroot-nix-store.service...551gitlab # [ 2.792379] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.552gitlab # [ 2.794008] systemd[1]: Stopped Virtual Console Setup.553gitlab # [ 2.795172] systemd[1]: Stopping Virtual Console Setup...554gitlab # [ 2.796551] systemd[1]: Starting Virtual Console Setup...555gitlab # [ 2.822375] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.556gitlab # [ 2.824078] systemd[1]: Finished rw-sysroot-nix-store.service.557gitlab # [ 2.838779] systemd-vconsole-setup[192]: Configuration of first virtual console was skipped, ignoring remaining ones.558gitlab # [ 2.841461] systemd[1]: Finished Virtual Console Setup.559gitlab # [ 3.380881] systemd[1]: Mounting /sysroot/nix/store...560gitlab # [ 3.427811] systemd[1]: Mounted /sysroot/nix/store.561gitlab # [ 3.429210] systemd[1]: Reached target Initrd File Systems.562gitlab # [ 3.430656] systemd[1]: Starting Find NixOS closure...563gitlab # [ 3.432766] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...564gitlab # [ 3.469805] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.565gitlab # [ 3.471851] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.566gitlab # [ 3.480796] systemd[1]: Finished Find NixOS closure.567gitlab # [ 3.482034] systemd[1]: Reached target Initrd Default Target.568gitlab # [ 3.483464] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...569gitlab # [ 3.516784] systemd[1]: Stopped target Initrd Default Target.570gitlab # [ 3.518203] systemd[1]: Stopped target Basic System.571gitlab # [ 3.519382] systemd[1]: Stopped target Initrd Root Device.572gitlab # [ 3.520745] systemd[1]: Stopped target Path Units.573gitlab # [ 3.521901] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.574gitlab # [ 3.523719] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.575gitlab # [ 3.525574] systemd[1]: Stopped target Slice Units.576gitlab # [ 3.526750] systemd[1]: Stopped target Socket Units.577gitlab # [ 3.529252] systemd[1]: Stopped target System Initialization.578gitlab # [ 3.530616] systemd[1]: Stopped target Swaps.579gitlab # [ 3.533029] systemd[1]: Stopped target Timer Units.580gitlab # [ 3.536762] systemd[1]: dbus.socket: Deactivated successfully.581gitlab # [ 3.539129] systemd[1]: Closed D-Bus System Message Bus Socket.582gitlab # [ 3.540857] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.583gitlab # [ 3.542601] systemd[1]: Stopped Find NixOS closure.584gitlab # [ 3.543767] systemd[1]: Starting Load Kernel Module 9pnet_virtio...585gitlab # [ 3.545280] systemd[1]: Starting rw-sysroot-nix-store.service...586gitlab # [ 3.546646] systemd[1]: systemd-sysctl.service: Deactivated successfully.587gitlab # [ 3.548453] systemd[1]: Stopped Apply Kernel Variables.588gitlab # [ 3.549959] systemd[1]: systemd-modules-load.service: Deactivated successfully.589gitlab # [ 3.551787] systemd[1]: Stopped Load Kernel Modules.590gitlab # [ 3.553045] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.591gitlab # [ 3.554824] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.592gitlab # [ 3.556882] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.593gitlab # [ 3.558632] systemd[1]: Stopped Create System Files and Directories.594gitlab # [ 3.560126] systemd[1]: Stopped target Local File Systems.595gitlab # [ 3.561408] systemd[1]: Stopped target Preparation for Local File Systems.596gitlab # [ 3.562885] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.597gitlab # [ 3.564496] systemd[1]: Stopped Coldplug All udev Devices.598gitlab # [ 3.565648] systemd[1]: Stopping Rule-based Manager for Device Events and Files...599gitlab # [ 3.567164] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.600gitlab # [ 3.568686] systemd[1]: Stopped Virtual Console Setup.601gitlab # [ 3.569737] systemd[1]: initrd-cleanup.service: Deactivated successfully.602gitlab # [ 3.571062] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.603gitlab # [ 3.572415] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.604gitlab # [ 3.573817] systemd[1]: Finished Load Kernel Module 9pnet_virtio.605gitlab # [ 3.575142] systemd[1]: systemd-udevd.service: Deactivated successfully.606gitlab # [ 3.576430] systemd[1]: Stopped Rule-based Manager for Device Events and Files.607gitlab # [ 3.577858] systemd[1]: systemd-udevd.service: Consumed 2.059s CPU time over 2.125s wall clock time, 31.8M memory peak.608gitlab # [ 3.579824] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.609gitlab # [ 3.581247] systemd[1]: Finished rw-sysroot-nix-store.service.610gitlab # [ 3.582348] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.611gitlab # [ 3.583665] systemd[1]: Closed udev Control Socket.612gitlab # [ 3.584655] systemd[1]: Starting Cleanup udev Database...613gitlab # [ 3.585675] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.614gitlab # [ 3.587067] systemd[1]: Stopped Create Static Device Nodes in /dev.615gitlab # [ 3.588293] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.616gitlab # [ 3.589762] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.617gitlab # [ 3.591076] systemd[1]: kmod-static-nodes.service: Deactivated successfully.618gitlab # [ 3.592403] systemd[1]: Stopped Create List of Static Device Nodes.619gitlab # [ 3.624627] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.620gitlab # [ 3.626372] systemd[1]: Finished Cleanup udev Database.621gitlab # [ 3.627683] systemd[1]: Reached target Switch Root.622gitlab # [ 3.628946] systemd[1]: Starting NixOS Activation...623gitlab # [ 3.753726] initrd-nixos-activation-start[218]: booting system configuration /nix/store/8sxiiab1vflpgf1s6qwyprgzkhnnrwlc-nixos-system-gitlab-test624gitlab # [ 3.791955] initrd-nixos-activation-start[218]: running activation script...625gitlab # [ 4.037911] initrd-nixos-activation-start[241]: setting up /etc...626gitlab # [ 4.159786] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.627gitlab # [ 4.161804] systemd[1]: Finished NixOS Activation.628gitlab # [ 4.163069] systemd[1]: Starting Switch Root...629gitlab # [ 4.196244] systemd[1]: Switching root.630gitlab # [ 4.336317] systemd-journald[96]: Received SIGTERM from PID 1 (systemd).631gitlab # [ 4.928505] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)632gitlab # [ 4.928634] systemd[1]: Detected virtualization qemu.633gitlab # [ 4.928729] systemd[1]: Detected architecture arm64.634gitlab # [ 4.928877] systemd[1]: Detected first boot.635gitlab # [ 4.930064] systemd[1]: Initializing machine ID from random generator.636gitlab # [ 5.268966] systemd[1]: bpf-restrict-fs: LSM BPF program attached637gitlab # [ 5.360678] systemd[1]: Applying preset policy.638gitlab # [ 5.523736] systemd[1]: Populated /etc with preset unit settings.639gitlab # [ 5.675841] systemd[1]: initrd-switch-root.service: Deactivated successfully.640gitlab # [ 5.676311] systemd[1]: Stopped initrd-switch-root.service.641gitlab # [ 5.679186] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.642gitlab # [ 5.682927] systemd[1]: Created slice Slice /system/getty.643gitlab # [ 5.685634] systemd[1]: Created slice GitLab DevOps Platform Slice.644gitlab # [ 5.688213] systemd[1]: Created slice User and Session Slice.645gitlab # [ 5.688674] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.646gitlab # [ 5.689476] systemd[1]: Started Forward Password Requests to Wall Directory Watch.647gitlab # [ 5.690095] systemd[1]: Expecting device /dev/hvc0...648gitlab # [ 5.690848] systemd[1]: Expecting device /dev/ttyAMA0...649gitlab # [ 5.691564] systemd[1]: Reached target Local Encrypted Volumes.650gitlab # [ 5.692272] systemd[1]: Stopped target initrd-fs.target.651gitlab # [ 5.693015] systemd[1]: Stopped target initrd-root-fs.target.652gitlab # [ 5.693707] systemd[1]: Stopped target initrd-switch-root.target.653gitlab # [ 5.694365] systemd[1]: Reached target Virtual Machines and Containers.654gitlab # [ 5.695050] systemd[1]: Reached target Path Units.655gitlab # [ 5.695706] systemd[1]: Reached target Remote File Systems.656gitlab # [ 5.696345] systemd[1]: Reached target Slice Units.657gitlab # [ 5.696997] systemd[1]: Reached target Swaps.658gitlab # [ 5.699418] systemd[1]: Listening on Query the User Interactively for a Password.659gitlab # [ 5.702018] systemd[1]: Listening on Process Core Dump Socket.660gitlab # [ 5.703958] systemd[1]: Listening on Credential Encryption/Decryption.661gitlab # [ 5.706146] systemd[1]: Listening on Factory Reset Management.662gitlab # [ 5.706990] systemd[1]: Listening on Hostname Service Socket.663gitlab # [ 5.712459] systemd[1]: Starting Journal Log Access Socket...664gitlab # [ 5.714167] systemd[1]: Listening on Journal Audit Socket.665gitlab # [ 5.716297] systemd[1]: Listening on Console Output Muting Service Socket.666gitlab # [ 5.716987] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.667gitlab # [ 5.717628] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os668gitlab # [ 5.718183] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki669gitlab # [ 5.721204] systemd[1]: Listening on Disk Repartitioning Service Socket.670gitlab # [ 5.721994] systemd[1]: Listening on udev Control Socket.671gitlab # [ 5.722710] systemd[1]: Listening on udev Varlink Socket.672gitlab # [ 5.727663] systemd[1]: Mounting Huge Pages File System...673gitlab # [ 5.734447] systemd[1]: Mounting POSIX Message Queue File System...674gitlab # [ 5.740946] systemd[1]: Mounting Kernel Debug File System...675gitlab # [ 5.746791] systemd[1]: Mounting Kernel Trace File System...676gitlab # [ 5.754030] systemd[1]: Starting Create List of Static Device Nodes...677gitlab # [ 5.764956] systemd[1]: Starting Load Kernel Module 9pnet_virtio...678gitlab # [ 5.767281] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs679gitlab # [ 5.775648] systemd[1]: Mounting Kernel Configuration File System...680gitlab # [ 5.777583] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm681gitlab # [ 5.778086] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore682gitlab # [ 5.785162] systemd[1]: Starting Load Kernel Module fuse...683gitlab # [ 5.787192] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67684gitlab # [ 5.798181] systemd[1]: Starting Journal Service...685gitlab # [ 5.805033] systemd[1]: Starting Load Kernel Modules...686gitlab # [ 5.813642] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...687gitlab # [ 5.822756] fuse: init (API version 7.45)688gitlab # [ 5.826050] systemd[1]: Starting Remount Root and Kernel File Systems...689gitlab # [ 5.826800] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os690gitlab # [ 5.838828] systemd[1]: Starting Coldplug All udev Devices...691gitlab # [ 5.846757] systemd-journald[312]: Collecting audit messages is enabled.692gitlab # [ 5.848013] systemd[1]: Listening on Journal Log Access Socket.693gitlab # [ 5.848649] systemd[1]: Mounted Huge Pages File System.694gitlab # [ 5.849169] systemd[1]: Mounted POSIX Message Queue File System.695gitlab # [ 5.849788] systemd[1]: Mounted Kernel Debug File System.696gitlab # [ 5.850326] systemd[1]: Mounted Kernel Trace File System.697gitlab # [ 5.851405] systemd[1]: Finished Create List of Static Device Nodes.698gitlab # [ 5.852745] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.699gitlab # [ 5.853972] systemd[1]: Finished Load Kernel Module 9pnet_virtio.700gitlab # [ 5.854719] systemd[1]: Mounted Kernel Configuration File System.701gitlab # [ 5.855617] systemd[1]: modprobe@fuse.service: Deactivated successfully.702gitlab # [ 5.856348] systemd[1]: Finished Load Kernel Module fuse.703gitlab # [ 5.857198] systemd[1]: Finished Load Kernel Modules.704gitlab # [ 5.865263] EXT4-fs (vda): re-mounted d82bc7c1-8ff8-443e-b559-388d38f5de9f.705gitlab # [ 5.865972] systemd[1]: Mounting FUSE Control File System...706gitlab # [ 5.869620] systemd[1]: Starting Firewall...707gitlab # [ 5.890772] systemd[1]: Queued start job for default target Multi-User System.708gitlab # [ 5.896539] systemd[1]: Unnecessary job was removed for /dev/vdb.709gitlab # [ 5.897863] systemd[1]: systemd-journald.service: Deactivated successfully.710gitlab # [ 5.899217] systemd-modules-load[313]: Using 3 probe threads711gitlab # [ 5.900447] systemd-modules-load[313]: Module 'atkbd' is built in712gitlab # [ 5.902257] systemd-modules-load[313]: Module 'loop' is built in[ 5.901476] systemd[1]: Starting Apply Kernel Variables...713gitlab # 714gitlab # [ 5.904286] systemd-modules-load[313]: Inserted module 'tls'715gitlab # [ 5.905954] systemd-oomd[315]: No swap; memory pressure usage will be degraded716gitlab # [ 5.908505] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...717gitlab # [ 5.916331] systemd[1]: Started Journal Service.718gitlab # [ 5.917526] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.719gitlab # [ 5.919465] systemd[1]: Finished Remount Root and Kernel File Systems.720gitlab # [ 5.922944] systemd[1]: Mounted FUSE Control File System.721gitlab # [ 5.924259] systemd[1]: Finished Apply Kernel Variables.722gitlab # [ 5.934330] systemd[1]: Listening on Disk Image Download Service Socket.723gitlab # [ 5.940363] systemd[1]: Starting Flush Journal to Persistent Storage...724gitlab # [ 5.941861] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore725gitlab # [ 5.947473] systemd[1]: Starting Load/Save OS Random Seed...726gitlab # [ 5.948705] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os727gitlab # [ 5.964257] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.728gitlab # [ 5.969813] systemd-journald[312]: Received client request to flush runtime journal.729gitlab # [ 5.997995] systemd[1]: Starting Create Static Device Nodes in /dev...730gitlab # [ 5.999952] systemd[1]: Finished Load/Save OS Random Seed.731gitlab # [ 6.001110] systemd[1]: Reached target First Boot Complete.732gitlab # [ 6.002039] systemd[1]: Finished Create Static Device Nodes in /dev.733gitlab # [ 6.002972] systemd[1]: Reached target Preparation for Local File Systems.734gitlab # [ 6.005140] systemd[1]: Starting Rule-based Manager for Device Events and Files...735gitlab # [ 6.006401] systemd[1]: Finished Flush Journal to Persistent Storage.736gitlab # [ 6.078977] systemd-udevd[361]: Using default interface naming scheme 'v261'.737gitlab # [ 6.115364] systemd[1]: Started Rule-based Manager for Device Events and Files.738gitlab # [ 6.268148] systemd[1]: Finished Firewall.739gitlab # [ 6.498022] systemd[1]: Finished Coldplug All udev Devices.740gitlab # [ 6.561327] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs741gitlab # [ 6.629394] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.742gitlab # [ 6.652854] (udev-worker)[423]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.743gitlab # [ 6.656699] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped.744gitlab # [ 6.658096] (udev-worker)[423]: Network interface NamePolicy= disabled on kernel command line.745gitlab # [ 6.666234] (udev-worker)[410]: Network interface NamePolicy= disabled on kernel command line.746gitlab # [ 6.689825] systemd[1]: Mounting /run/wrappers...747gitlab # [ 6.697429] systemd[1]: Condition check resulted in Virtio network device being skipped.748gitlab # [ 6.699525] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore749gitlab # [ 6.701648] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67750gitlab # [ 6.703931] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore751gitlab # [ 6.706131] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os752gitlab # [ 6.708281] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os753gitlab # [ 6.719554] systemd[1]: Mounted /run/wrappers.754gitlab # [ 6.720676] systemd[1]: Reached target Local File Systems.755gitlab # [ 6.721925] systemd[1]: Listening on Boot Loader Control Service Socket.756gitlab # [ 6.724859] systemd[1]: Starting register-nix-paths.service...757gitlab # [ 6.732531] systemd[1]: Starting Create SUID/SGID Wrappers...758gitlab # [ 6.734635] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.759gitlab # [ 6.736303] systemd[1]: Starting Save Transient machine-id to Disk...760gitlab # [ 6.752495] systemd[1]: Starting Create System Files and Directories...761gitlab # [ 6.780134] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.762gitlab # [ 6.783507] systemd[1]: Finished Save Transient machine-id to Disk.763gitlab # [ 6.796859] systemd[1]: Finished Create System Files and Directories.764gitlab # [ 6.804412] systemd[1]: Starting Rebuild Journal Catalog...765gitlab # [ 6.806408] systemd[1]: Starting Record System Boot/Shutdown in UTMP...766gitlab # [ 6.827113] mousedev: PS/2 mouse device common for all mice767gitlab # [ 6.847139] systemd[1]: Finished Record System Boot/Shutdown in UTMP.768gitlab # [ 6.851731] systemd[1]: Finished Rebuild Journal Catalog.769gitlab # [ 6.856617] systemd[1]: Starting Update is Completed...770gitlab # [ 6.897051] systemd[1]: Finished Update is Completed.771gitlab # [ 6.976459] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.772gitlab # [ 6.978088] systemd[1]: Finished Create SUID/SGID Wrappers.773gitlab # [ 6.980690] systemd[1]: suid-sgid-wrappers.service: Consumed 168ms CPU time over 243ms wall clock time, 6.3M memory peak, 1.9M read from disk.774gitlab # [ 7.226736] systemd[1]: Finished register-nix-paths.service.775gitlab # [ 7.228916] systemd[1]: Reached target System Initialization.776gitlab # [ 7.230897] systemd[1]: Started Discard unused filesystem blocks once a week.777gitlab # [ 7.232874] systemd[1]: Started logrotate.timer.778gitlab # [ 7.234525] systemd[1]: Started Daily Cleanup of Temporary Directories.779gitlab # [ 7.235611] systemd[1]: Reached target Timer Units.780gitlab # [ 7.236489] systemd[1]: Listening on D-Bus System Message Bus Socket.781gitlab # [ 7.237697] systemd[1]: Listening on Nix Daemon Socket.782gitlab # [ 7.242810] systemd[1]: Starting nixbot CI service socket...783gitlab # [ 7.243864] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.784gitlab # [ 7.245950] systemd[1]: Listening on nixbot CI service socket.785gitlab # [ 7.247189] systemd[1]: Reached target Socket Units.786gitlab # [ 7.249900] systemd[1]: Starting D-Bus System Message Bus...787gitlab # [ 7.285438] dbus-broker-launch[555]: Looking up NSS user entry for 'systemd-timesync'...788gitlab # [ 7.287181] dbus-broker-launch[555]: NSS returned no entry for 'systemd-timesync'789gitlab # [ 7.288515] dbus-broker-launch[555]: Invalid user-name in /nix/store/9zcyylhndlnizbwn48627991zr1jxg4p-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"790gitlab # [ 7.293152] systemd[1]: Started D-Bus System Message Bus.791gitlab # [ 7.303789] dbus-broker-launch[555]: Ready792gitlab # [ 7.305937] systemd[1]: Reached target Basic System.793gitlab # [ 7.309051] systemd[1]: Started backdoor.service.794gitlab # [ 7.314629] systemd[1]: Starting gitlab-config.service...795gitlab # [ 7.337405] systemd[1]: Starting Import lastlog data into lastlog2 database...796gitlab # [ 7.348375] systemd[1]: Starting Logrotate configuration check...797gitlab # [ 7.354214] systemd[1]: Starting Name Service Cache Daemon (nsncd)...798gitlab # [ 7.357323] systemd[1]: Starting Post-Boot Actions...799gitlab # [ 7.367319] systemd[1]: Started Reset console on configuration changes.800gitlab # [ 7.380268] logrotate[562]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want.801gitlab # connecting to host...802gitlab # [ 7.383079] logrotate[562]: reading config file /nix/store/22afkmbkaxyx50bj9lbkkpmz7pd69hsf-logrotate.conf803gitlab # [ 7.384957] logrotate[562]: note: 'monthly' overrides previously specified 'weekly'804gitlab # [ 7.386046] logrotate[562]: Reading state from file: /var/lib/logrotate.status805gitlab # [ 7.387094] logrotate[562]: state file /var/lib/logrotate.status does not exist806gitlab # [ 7.388592] logrotate[562]: Allocating hash table for state file, size 64 entries807gitlab # [ 7.390853] logrotate[562]: Handling 4 logs808gitlab # [ 7.392685] logrotate[562]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed809gitlab # [ 7.398027] logrotate[562]: considering log /var/log/btmp810gitlab: Guest shell says: b'Spawning backdoor root shell...\n'811gitlab # [ 7.398910] logrotate[562]: Creating new state812gitlab # [ 7.399685] logrotate[562]: Now: 2026-10-02 14:09813gitlab # [ 7.401547] logrotate[562]: Last rotated at 2026-10-02 14:00814gitlab: connected to guest root shell815gitlab # [ 7.402347] logrotate[562]: log does not need rotating (log has already been rotated)816gitlab: (connecting took 7.85 seconds)817gitlab # [ 7.403501] logrotate[562]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed818gitlab: (finished: waiting for the VM to finish booting, in 7.85 seconds)819gitlab # [ 7.405684] logrotate[562]: considering log /var/log/wtmp820gitlab # [ 7.406380] logrotate[562]: Creating new state821gitlab # [ 7.406993] logrotate[562]: Now: 2026-10-02 14:09822gitlab # [ 7.407637] logrotate[562]: Last rotated at 2026-10-02 14:00823gitlab # [ 7.408431] logrotate[562]: log does not need rotating (log has already been rotated)824gitlab # [ 7.409413] logrotate[562]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed825gitlab # [ 7.411085] logrotate[562]: switching euid from 0 to 165 and egid from 0 to 165 (pid 562)826gitlab # [ 7.412184] logrotate[562]: considering log /var/gitlab/state/log/*.log827gitlab # [ 7.413012] logrotate[562]: log /var/gitlab/state/log/*.log does not exist -- skipping828gitlab # [ 7.414003] logrotate[562]: Creating new state829gitlab # [ 7.414584] logrotate[562]: switching euid from 165 to 0 and egid from 165 to 0 (pid 562)830gitlab # [ 7.415592] logrotate[562]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed831gitlab # [ 7.417353] logrotate[562]: switching euid from 0 to 60 and egid from 0 to 60 (pid 562)832gitlab # [ 7.418338] logrotate[562]: considering log /var/log/nginx/*.log833gitlab # [ 7.419094] logrotate[562]: log /var/log/nginx/*.log does not exist -- skipping834gitlab # [ 7.420228] logrotate[562]: Creating new state835gitlab # [ 7.420855] logrotate[562]: not running postrotate script, since no logs were rotated836gitlab # [ 7.421980] logrotate[562]: switching euid from 60 to 0 and egid from 60 to 0 (pid 562)837gitlab # [ 7.423458] systemd[1]: Starting resolvconf update...838gitlab # [ 7.428764] nsncd[565]: Oct 02 14:09:25.312 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"839gitlab # [ 7.446061] systemd[1]: Started Name Service Cache Daemon (nsncd).840gitlab # [ 7.480997] systemd[1]: Finished Logrotate configuration check.841gitlab # [ 7.496488] systemd[1]: Finished Post-Boot Actions.842gitlab # [ 7.513460] systemd[1]: Finished Import lastlog data into lastlog2 database.843gitlab # [ 7.542188] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:09.0/virtio8/input/input3844gitlab # [ 7.559957] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...845gitlab # [ 7.579408] systemd[1]: Starting Virtual Console Setup...846gitlab # [ 7.580539] systemd[1]: nscd.service: Deactivated successfully.847gitlab # [ 7.581403] systemd[1]: Stopped Name Service Cache Daemon (nsncd).848gitlab # [ 7.582299] systemd[1]: nscd.service: Consumed 33ms CPU time over 228ms wall clock time, 7.7M memory peak, 5.8M read from disk.849gitlab # [ 7.594257] systemd[1]: Finished resolvconf update.850gitlab # [ 7.602675] systemd[1]: Reached target Preparation for Network.851gitlab # [ 7.608262] systemd[1]: Starting DHCP Client...852gitlab # [ 7.614872] systemd[1]: Starting Address configuration of eth1...853gitlab # [ 7.639010] systemd[1]: Starting Extra networking commands....854gitlab # [ 7.644111] systemd[1]: Starting Name Service Cache Daemon (nsncd)...855gitlab # [ 7.653232] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.856gitlab # [ 7.654635] systemd[1]: Stopped Virtual Console Setup.857gitlab # [ 7.656163] systemd[1]: systemd-vconsole-setup.service: Consumed 41ms CPU time over 73ms wall clock time, 3M memory peak, 1.1M read from disk.858gitlab # [ 7.675921] nsncd[669]: Oct 02 14:09:25.585 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"859gitlab # [ 7.682697] network-addresses-eth1-start[664]: adding address 192.168.1.1/24... done860gitlab # [ 7.683911] systemd[1]: Starting Virtual Console Setup...861gitlab # [ 7.688687] systemd[1]: Started Name Service Cache Daemon (nsncd).862gitlab # [ 7.689588] network-addresses-eth1-start[664]: adding address 2001:db8:1::1/64... done863gitlab # [ 7.690663] systemd[1]: Reached target Host and Network Name Lookups.864gitlab # [ 7.691672] systemd[1]: Reached target User and Group Name Lookups.865gitlab # [ 7.705186] systemd[1]: Starting User Login Management...866gitlab # [ 7.710658] systemd[1]: Finished Address configuration of eth1.867gitlab # [ 7.733024] systemd-vconsole-setup[691]: Configuration of first virtual console was skipped, ignoring remaining ones.868gitlab # [ 7.744314] systemd[1]: Finished Virtual Console Setup.869gitlab # [ 7.772518] dhcpcd[674]: dhcpcd-10.3.2 starting870gitlab # [ 7.773470] systemd[1]: Finished Extra networking commands..871gitlab # [ 7.774420] systemd[1]: Reached target Network.872gitlab # [ 7.777474] systemd-logind[709]: New seat seat0.873gitlab # [ 7.782073] systemd[1]: Starting gitlab-workhorse.service...874gitlab # [ 7.785400] dhcpcd[752]: dev: loaded udev875gitlab # [ 7.786134] systemd-logind[709]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard)876gitlab # [ 7.787745] systemd-logind[709]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard)877gitlab # [ 7.789349] systemd-logind[709]: Watching system buttons on /dev/input/event0 (gpio-keys)878gitlab # [ 7.790568] systemd[1]: Starting Nginx Web Server...879gitlab # [ 7.795727] systemd[1]: Starting PostgreSQL Server...880gitlab # [ 7.801194] 8021q: 802.1Q VLAN Support v1.8881gitlab # [ 7.801855] 8021q: adding VLAN 0 to HW filter on device eth1882gitlab # [ 7.804178] systemd[1]: Starting Redis Server - redis-gitlab...883gitlab # [ 7.809051] systemd[1]: Starting Permit User Sessions...884gitlab # [ 7.810059] systemd[1]: Started User Login Management.885gitlab # [ 7.852139] systemd[1]: Starting linger-users.service...886gitlab # [ 7.855007] cfg80211: Loading compiled-in X.509 certificates for regulatory database887gitlab # [ 7.857820] systemd[1]: Finished Permit User Sessions.888gitlab # [ 7.873952] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.[ 7.874064] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'889gitlab # [ 7.874546] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'890gitlab # [ 7.877848] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2891gitlab # [ 7.878206] cfg80211: failed to load regulatory.db892gitlab # 893gitlab # [ 7.895585] systemd[1]: Started Getty on tty1.894gitlab # [ 7.897317] systemd[1]: Reached target Login Prompts.895gitlab # [ 7.898069] systemd[1]: linger-users.service: Deactivated successfully.896gitlab # [ 7.899302] systemd[1]: Finished linger-users.service.897gitlab # [ 7.934707] 8021q: adding VLAN 0 to HW filter on device eth0898gitlab # [ 7.934083] dhcpcd[752]: eth0: waiting for carrier899gitlab # [ 7.936345] dhcpcd[752]: eth0: carrier acquired900gitlab # [ 7.941696] dhcpcd[752]: DUID 00:01:00:01:32:52:77:95:52:54:00:12:34:56901gitlab # [ 7.942873] dhcpcd[752]: eth0: IAID 00:12:34:56902gitlab # [ 7.943710] dhcpcd[752]: eth0: adding address fe80::5054:ff:fe12:3456903gitlab # [ 8.013161] nginx-pre-start[790]: nginx: the configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf syntax is ok904gitlab # [ 8.015171] nginx-pre-start[790]: nginx: configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf test is successful905gitlab # [ 8.025087] systemd[1]: Started Nginx Web Server.906gitlab # [ 8.095338] redis-server[788]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.907gitlab # [ 8.100291] redis-server[788]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo908gitlab # [ 8.102810] redis-server[788]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=788, just started909gitlab # [ 8.104851] redis-server[788]: Configuration loaded910gitlab # [ 8.105785] redis[788]: monotonic clock: ARM CNTVCT @ 1000 ticks/us911gitlab # [ 8.106945] redis[788]: Running mode=standalone, port=0.912gitlab # [ 8.109532] redis[788]: Server initialized913gitlab # [ 8.110918] systemd[1]: Started Redis Server - redis-gitlab.914gitlab # [ 8.114663] postgresql-pre-start[793]: The files belonging to this database system will be owned by user "postgres".915gitlab # [ 8.116392] postgresql-pre-start[793]: This user must also own the server process.916gitlab # [ 8.117689] postgresql-pre-start[793]: The database cluster will be initialized with locale "en_US.UTF-8".917gitlab # [ 8.119407] postgresql-pre-start[793]: The default database encoding has accordingly been set to "UTF8".918gitlab # [ 8.120993] postgresql-pre-start[793]: The default text search configuration will be set to "english".919gitlab # [ 8.122312] postgresql-pre-start[793]: Data page checksums are enabled.920gitlab # [ 8.123458] postgresql-pre-start[793]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok921gitlab # [ 8.125597] postgresql-pre-start[793]: creating subdirectories ... ok922gitlab # [ 8.126622] postgresql-pre-start[793]: selecting dynamic shared memory implementation ... posix923gitlab # [ 8.127998] redis[788]: Ready to accept connections unix924gitlab # [ 8.129064] redis[788]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.925gitlab # [ 8.168430] postgresql-pre-start[793]: selecting default "max_connections" ... 100926gitlab # [ 8.226623] postgresql-pre-start[793]: selecting default "shared_buffers" ... 128MB927gitlab # [ 8.586681] systemd[1]: Started gitlab-workhorse.service.928gitlab # [ 8.732943] postgresql-pre-start[793]: selecting default time zone ... UTC929gitlab # [ 8.735094] postgresql-pre-start[793]: creating configuration files ... ok930gitlab # [ 8.966185] gitlab-workhorse[816]: time="2026-10-02T14:09:26Z" level=info msg=Starting build_time=19700101.000000 version=19.0.4931gitlab # [ 8.969418] gitlab-workhorse[816]: time="2026-10-02T14:09:26Z" level=info msg="Using redis/go-redis"932gitlab # [ 8.980403] gitlab-workhorse[816]: time="2026-10-02T14:09:26Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix933gitlab # [ 9.016967] postgresql-pre-start[793]: running bootstrap script ... ok934gitlab # [ 9.175496] dhcpcd[752]: eth0: soliciting an IPv6 router935gitlab # [ 9.177135] dhcpcd[752]: eth0: Router Advertisement from fe80::2936gitlab # [ 9.178670] dhcpcd[752]: eth0: adding address fec0::5054:ff:fe12:3456/64937gitlab # [ 9.180433] dhcpcd[752]: eth0: adding route to fec0::/64938gitlab # [ 9.181812] dhcpcd[752]: eth0: adding default route via fe80::2939gitlab # [ 9.491680] dhcpcd[752]: eth0: soliciting a DHCP lease940gitlab # [ 9.500667] dhcpcd[752]: eth0: offered 10.0.2.15 from 10.0.2.2941gitlab # [ 9.516352] dhcpcd[752]: eth0: probing address 10.0.2.15/24942gitlab # [ 9.778477] postgresql-pre-start[793]: performing post-bootstrap initialization ... ok943gitlab # [ 10.004823] postgresql-pre-start[793]: syncing data to disk ... ok944gitlab # [ 10.006451] postgresql-pre-start[793]: initdb: warning: enabling "trust" authentication for local connections945gitlab # [ 10.008831] postgresql-pre-start[793]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.946gitlab # [ 10.012577] postgresql-pre-start[793]: Success. You can now start the database server using:947gitlab # [ 10.014684] postgresql-pre-start[793]: pg_ctl -D /var/lib/postgresql/18 -l logfile start948gitlab # [ 10.136911] postgres[850]: [850] LOG: starting PostgreSQL 18.4 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit949gitlab # [ 10.140191] postgres[850]: [850] LOG: listening on IPv6 address "::1", port 5432950gitlab # [ 10.142113] postgres[850]: [850] LOG: listening on IPv4 address "127.0.0.1", port 5432951gitlab # [ 10.144086] postgres[850]: [850] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"952gitlab # [ 10.153883] postgres[859]: [859] LOG: database system was shut down at 2026-10-02 14:09:27 GMT953gitlab # [ 10.161301] postgres[850]: [850] LOG: database system is ready to accept connections954gitlab # [ 10.167317] systemd[1]: Started PostgreSQL Server.955gitlab # [ 10.174164] systemd[1]: Starting PostgreSQL Setup Scripts...956gitlab # [ 10.461638] postgresql-setup-start[870]: CREATE DATABASE957gitlab # [ 10.499684] postgresql-setup-start[880]: CREATE ROLE958gitlab # [ 10.516943] postgresql-setup-start[882]: ALTER DATABASE959gitlab # [ 10.563925] postgresql-setup-start[887]: CREATE ROLE960gitlab # [ 10.567783] systemd[1]: Finished PostgreSQL Setup Scripts.961gitlab # [ 10.569532] systemd[1]: Reached target PostgreSQL.962gitlab # [ 10.573531] systemd[1]: Starting gitlab-postgresql.service...963gitlab # [ 10.752291] gitlab-postgresql-start[898]: CREATE DATABASE964gitlab # [ 10.799704] gitlab-postgresql-start[908]: CREATE EXTENSION965gitlab # [ 10.826889] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[899]: mkdir -p /var/gitlab/state/home/.ssh: OK966gitlab # [ 10.829490] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[899]: chmod 700 /var/gitlab/state/home/.ssh: OK967gitlab # [ 10.896933] gitlab-postgresql-start[915]: CREATE EXTENSION968gitlab # [ 10.901241] systemd[1]: Finished gitlab-config.service.969gitlab # [ 10.902562] systemd[1]: Finished gitlab-postgresql.service.970gitlab # [ 10.907884] systemd[1]: Started gitaly.service.971gitlab # [ 10.912340] systemd[1]: Starting gitlab-db-config.service...972gitlab # [ 11.653625] gitaly[930]: time="2026-10-02T14:09:29.563Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/0r0qd84zrxhqq373hk9spv8j9pg0qlnq-gitaly.toml' and fix the errors that are printed." pid=930973gitlab # [ 11.653925] gitaly[930]: time="2026-10-02T14:09:29.563Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=930974gitlab # [ 11.653990] gitaly[930]: time="2026-10-02T14:09:29.563Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=930975gitlab # [ 11.654040] gitaly[930]: time="2026-10-02T14:09:29.563Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=930976gitlab # [ 11.654090] gitaly[930]: time="2026-10-02T14:09:29.563Z" level=info msg="Starting Gitaly" pid=930 version=977gitlab # [ 11.941939] gitaly[930]: time="2026-10-02T14:09:29.851Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=287 pid=930978gitlab # [ 11.944642] gitaly[930]: time="2026-10-02T14:09:29.852Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=930979gitlab # [ 11.946687] gitaly[930]: time="2026-10-02T14:09:29.852Z" level=info msg="finished initializing command factory" duration_ms=0 pid=930980gitlab # [ 11.949016] gitaly[930]: time="2026-10-02T14:09:29.852Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-930/git-exec-2978013136.d/git pid=930981gitlab # [ 11.952442] gitaly[930]: time="2026-10-02T14:09:29.862Z" level=info msg="finished detecting git version" duration_ms=9 pid=930982gitlab # [ 11.954538] gitaly[930]: time="2026-10-02T14:09:29.862Z" level=info msg="using Git version" pid=930 version=2.53.g983gitlab # [ 11.956599] gitaly[930]: time="2026-10-02T14:09:29.862Z" level=info msg="clearing disk cache object folder" pid=930 storage=default984gitlab # [ 11.958759] gitaly[930]: time="2026-10-02T14:09:29.863Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3730575983 pid=930 storage=default985gitlab # [ 11.962163] gitaly[930]: time="2026-10-02T14:09:29.863Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3730575983 pid=930 storage=default986gitlab # [ 11.965991] gitaly[930]: time="2026-10-02T14:09:29.863Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=930987gitlab # [ 11.969593] gitaly[930]: time="2026-10-02T14:09:29.863Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=930988gitlab # [ 11.972821] gitaly[930]: time="2026-10-02T14:09:29.863Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3730575983 clear_duration_ms=0 pid=930 storage=default989gitlab # [ 11.976682] gitaly[930]: time="2026-10-02T14:09:29.880Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=930990gitlab # [ 11.979914] gitaly[930]: time="2026-10-02T14:09:29.880Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=930 schema=unix991gitlab # [ 11.982746] gitaly[930]: time="2026-10-02T14:09:29.880Z" level=info msg="listening at address" address=/run/gitaly/gitaly-930/sock.d/intern pid=930 schema=unix992gitlab # [ 11.985614] gitaly[930]: time="2026-10-02T14:09:29.880Z" level=info msg="counting repositories" pid=930993gitlab # [ 11.987319] gitaly[930]: time="2026-10-02T14:09:29.881Z" level=info msg="starting to count repositories" pid=930 storage_path=/var/gitlab/state/repositories994gitlab # [ 11.989949] gitaly[930]: time="2026-10-02T14:09:29.881Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=930 storage_path=/var/gitlab/state/repositories995gitlab # [ 11.992884] gitaly[930]: time="2026-10-02T14:09:29.881Z" level=info msg="completed counting all repositories" counting_duration=0 pid=930996gitlab # [ 11.995300] gitaly[930]: time="2026-10-02T14:09:29.881Z" level=info msg="finished tempdir cleaner walk" pid=930 storage=default time_ms=0997gitlab # [ 11.997746] gitaly[930]: time="2026-10-02T14:09:29.888Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=930998gitlab # [ 12.000181] gitaly[930]: time="2026-10-02T14:09:29.888Z" level=info msg="Started Gitaly" duration_ms=324 pid=930999gitlab # [ 12.002056] gitaly[930]: time="2026-10-02T14:09:29.888Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=9301000gitlab # [ 12.004325] gitaly[930]: time="2026-10-02T14:09:29.888Z" level=info msg="maintenance: daily scheduled" pid=930 scheduled="2026-10-03 12:00:00 +0000 UTC"1001gitlab # [ 12.098411] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[933]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1002gitlab # [ 12.102389] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[933]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1003gitlab # [ 12.116060] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[933]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1004gitlab # [ 14.375598] dhcpcd[752]: eth0: leased 10.0.2.15 for 86400 seconds1005gitlab # [ 14.375873] dhcpcd[752]: eth0: adding route to 10.0.2.0/241006gitlab # [ 14.375970] dhcpcd[752]: eth0: adding default route via 10.0.2.21007gitlab # [ 14.513050] systemd[1]: Started DHCP Client.1008gitlab # [ 14.515618] systemd[1]: Reached target Network is Online.1009gitlab # [ 15.481092] gitaly[930]: time="2026-10-02T14:09:33.390Z" level=info msg="License database preloaded" duration_ms=3826 pid=9301010gitlab # [ 72.472131] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[933]: W, [2026-10-02T14:10:30.381404 #933] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1011gitlab # [ 76.405392] postgres[1581]: [1581] ERROR: relation "feature_gates" does not exist at character 5231012gitlab # [ 76.405717] postgres[1581]: [1581] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod),1013gitlab # [ 76.405807] postgres[1581]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1014gitlab # [ 76.405909] postgres[1581]: c.collname, col_description(a.attrelid, a.attnum) AS comment,1015gitlab # [ 76.405986] postgres[1581]: attidentity AS identity,1016gitlab # [ 76.406065] postgres[1581]: attgenerated as attgenerated1017gitlab # [ 76.406145] postgres[1581]: FROM pg_attribute a1018gitlab # [ 76.406224] postgres[1581]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1019gitlab # [ 76.406307] postgres[1581]: LEFT JOIN pg_type t ON a.atttypid = t.oid1020gitlab # [ 76.406385] postgres[1581]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1021gitlab # [ 76.406463] postgres[1581]: WHERE a.attrelid = '"feature_gates"'::regclass1022gitlab # [ 76.406545] postgres[1581]: AND a.attnum > 0 AND NOT a.attisdropped1023gitlab # [ 76.406650] postgres[1581]: ORDER BY a.attnum1024gitlab # [ 76.406744] postgres[1581]:1025gitlab # [ 79.059973] gitlab-workhorse[816]: time="2026-10-02T14:10:36Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1026gitlab # [ 82.478190] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1641]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping1027gitlab # [ 82.478570] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1641]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping1028gitlab # [ 89.069565] gitlab-workhorse[816]: time="2026-10-02T14:10:46Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1029gitlab # [ 99.071051] gitlab-workhorse[816]: time="2026-10-02T14:10:56Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1030gitlab # [ 107.558064] postgres[1762]: [1762] ERROR: relation "table_without_models" does not exist1031gitlab # [ 107.558534] postgres[1762]: [1762] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1032gitlab # [ 109.081016] gitlab-workhorse[816]: time="2026-10-02T14:11:06Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1033gitlab # [ 119.082183] gitlab-workhorse[816]: time="2026-10-02T14:11:16Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1034gitlab # [ 128.615150] systemd[1]: Finished gitlab-db-config.service.1035gitlab # [ 128.623735] systemd[1]: Started gitlab-sidekiq.service.1036gitlab # [ 128.625521] systemd[1]: Starting gitlab.service...1037gitlab # [ 129.084078] gitlab-workhorse[816]: time="2026-10-02T14:11:26Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1038gitlab # [ 129.275301] bundle[2046]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1039gitlab # [ 129.279033] bundle[2046]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1040gitlab # [ 129.282380] bundle[2046]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1041gitlab # [ 137.945681] sidekiq-cluster[2045]: {"severity":"INFO","time":"2026-10-02T14:11:35.811Z","message":"Starting cluster with 1 processes"}1042gitlab # [ 137.961865] bundle[2046]: {"timestamp":"2026-10-02T14:11:35.827Z","pid":2046,"message":"* Enabling systemd notification integration"}1043gitlab # [ 137.964723] bundle[2046]: {"timestamp":"2026-10-02T14:11:35.871Z","pid":2046,"message":"Puma starting in cluster mode..."}1044gitlab # [ 137.967101] bundle[2046]: {"timestamp":"2026-10-02T14:11:35.871Z","pid":2046,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"}1045gitlab # [ 137.969761] bundle[2046]: {"timestamp":"2026-10-02T14:11:35.871Z","pid":2046,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1046gitlab # [ 137.973028] bundle[2046]: {"timestamp":"2026-10-02T14:11:35.871Z","pid":2046,"message":"* Min threads: 0"}1047gitlab # [ 137.975129] bundle[2046]: {"timestamp":"2026-10-02T14:11:35.871Z","pid":2046,"message":"* Max threads: 4"}1048gitlab # [ 137.977290] bundle[2046]: {"timestamp":"2026-10-02T14:11:35.871Z","pid":2046,"message":"* Environment: production"}1049gitlab # [ 137.979611] bundle[2046]: {"timestamp":"2026-10-02T14:11:35.871Z","pid":2046,"message":"* Master PID: 2046"}1050gitlab # [ 137.981875] bundle[2046]: {"timestamp":"2026-10-02T14:11:35.871Z","pid":2046,"message":"* Workers: 2"}1051gitlab # [ 137.983936] bundle[2046]: {"timestamp":"2026-10-02T14:11:35.871Z","pid":2046,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"}1052gitlab # [ 137.986809] bundle[2046]: {"timestamp":"2026-10-02T14:11:35.871Z","pid":2046,"message":"* Preloading application"}1053gitlab # [ 139.093481] gitlab-workhorse[816]: time="2026-10-02T14:11:37Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1054gitlab # [ 149.104723] gitlab-workhorse[816]: time="2026-10-02T14:11:47Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1055gitlab # [ 159.105822] gitlab-workhorse[816]: time="2026-10-02T14:11:57Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1056gitlab # [ 169.116172] gitlab-workhorse[816]: time="2026-10-02T14:12:07Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1057gitlab # [ 179.125775] gitlab-workhorse[816]: time="2026-10-02T14:12:17Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1058gitlab # [ 189.134228] gitlab-workhorse[816]: time="2026-10-02T14:12:27Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1059gitlab # [ 199.135083] gitlab-workhorse[816]: time="2026-10-02T14:12:37Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1060gitlab # [ 206.661613] bundle[2046]: W, [2026-10-02T14:12:44.571019 #2046] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1061gitlab # [ 209.141691] gitlab-workhorse[816]: time="2026-10-02T14:12:47Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1062gitlab # [ 214.461565] sidekiq-cluster[2132]: 2026-10-02T14:12:52.370Z pid=2132 tid=xrw INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2132, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>431312896}1063gitlab # [ 215.116935] sidekiq-cluster[2132]: W, [2026-10-02T14:12:53.026829 #2132] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1064gitlab # [ 215.298978] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:53.208Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1065gitlab # [ 219.151550] gitlab-workhorse[816]: time="2026-10-02T14:12:57Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1066gitlab # [ 219.992387] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:57.901Z","message":"GitLab reliable fetch activated!"}1067gitlab # [ 219.992668] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:57.901Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1068gitlab # [ 220.030069] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:57.940Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1069gitlab # [ 220.035867] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:57.945Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"}1070gitlab # [ 220.041012] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:57.951Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1071gitlab # [ 220.045143] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:57.955Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1072gitlab # [ 220.049568] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:57.959Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1073gitlab # [ 220.053374] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:57.963Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1074gitlab # [ 220.057219] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:57.965Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1075gitlab # [ 220.060694] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:57.968Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1076gitlab # [ 220.064144] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:57.970Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1077gitlab # [ 220.067693] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:57.973Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1078gitlab # [ 220.071802] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:57.975Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1079gitlab # [ 220.075835] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:57.978Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1080gitlab # [ 220.079510] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:57.981Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1081gitlab # [ 220.083331] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:57.984Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1082gitlab # [ 220.087080] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:57.988Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1083gitlab # [ 220.090635] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:57.992Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1084gitlab # [ 220.093885] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:57.997Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1085gitlab # [ 220.096907] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.001Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1086gitlab # [ 220.100138] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.006Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1087gitlab # [ 220.103119] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.011Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1088gitlab # [ 220.106483] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.015Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1089gitlab # [ 220.109828] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.019Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1090gitlab # [ 220.112639] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.022Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1091gitlab # [ 220.115575] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.025Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1092gitlab # [ 220.119347] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.029Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1093gitlab # [ 220.123774] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.033Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1094gitlab # [ 220.127754] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.037Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1095gitlab # [ 220.133232] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.043Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1096gitlab # [ 220.137595] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.047Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1097gitlab # [ 220.141763] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.051Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1098gitlab # [ 220.145613] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.055Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1099gitlab # [ 220.149625] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.059Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1100gitlab # [ 220.153881] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.063Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1101gitlab # [ 220.157255] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.066Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1102gitlab # [ 220.160526] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.069Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1103gitlab # [ 220.163747] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.072Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1104gitlab # [ 220.167042] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.075Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1105gitlab # [ 220.170140] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.079Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1106gitlab # [ 220.173057] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.082Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1107gitlab # [ 220.175723] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.085Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1108gitlab # [ 220.178751] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.088Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1109gitlab # [ 220.181412] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.090Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1110gitlab # [ 220.183967] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.093Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1111gitlab # [ 220.186622] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.096Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1112gitlab # [ 220.189305] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.099Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1113gitlab # [ 220.191883] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.101Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1114gitlab # [ 220.194485] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.104Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1115gitlab # [ 220.196991] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.107Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1116gitlab # [ 220.200038] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.110Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1117gitlab # [ 220.202823] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.112Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1118gitlab # [ 220.205591] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.115Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1119gitlab # [ 220.208895] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.119Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1120gitlab # [ 220.221597] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.131Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1121gitlab # [ 220.225226] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.135Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1122gitlab # [ 220.228188] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.138Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1123gitlab # [ 220.231178] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.141Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1124gitlab # [ 220.234092] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.144Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1125gitlab # [ 220.237043] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.147Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1126gitlab # [ 220.239810] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.149Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1127gitlab # [ 220.242786] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.151Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1128gitlab # [ 220.245362] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.154Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1129gitlab # [ 220.247718] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.157Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1130gitlab # [ 220.250333] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.159Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1131gitlab # [ 220.252868] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.162Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1132gitlab # [ 220.255354] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.165Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1133gitlab # [ 220.258418] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.168Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1134gitlab # [ 220.260808] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.170Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1135gitlab # [ 220.263205] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.172Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1136gitlab # [ 220.265919] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.175Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1137gitlab # [ 220.268620] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.177Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1138gitlab # [ 220.271046] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.179Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1139gitlab # [ 220.273577] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.181Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1140gitlab # [ 220.275988] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.183Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1141gitlab # [ 220.278564] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.185Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1142gitlab # [ 220.281132] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.188Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1143gitlab # [ 220.283676] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.190Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1144gitlab # [ 220.286204] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.192Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1145gitlab # [ 220.288905] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.194Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1146gitlab # [ 220.291286] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.197Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1147gitlab # [ 220.293712] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.199Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1148gitlab # [ 220.296221] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.201Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1149gitlab # [ 220.298771] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.203Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1150gitlab # [ 220.301305] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.207Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1151gitlab # [ 220.303735] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.209Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1152gitlab # [ 220.306900] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.212Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1153gitlab # [ 220.309471] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.215Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1154gitlab # [ 220.312103] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.218Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1155gitlab # [ 220.314679] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.221Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1156gitlab # [ 220.317365] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.223Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1157gitlab # [ 220.319899] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.226Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1158gitlab # [ 220.322686] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.228Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1159gitlab # [ 220.325271] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.230Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1160gitlab # [ 220.327842] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.232Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1161gitlab # [ 220.330792] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.235Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1162gitlab # [ 220.333163] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.238Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1163gitlab # [ 220.335790] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.242Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1164gitlab # [ 220.338594] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.244Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1165gitlab # [ 220.341212] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.247Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1166gitlab # [ 220.351680] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.261Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1167gitlab # [ 220.354833] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.264Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1168gitlab # [ 220.358379] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.267Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1169gitlab # [ 220.362556] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.272Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1170gitlab # [ 220.366506] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.276Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1171gitlab # [ 220.370014] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.280Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1172gitlab # [ 220.373736] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.283Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1173gitlab # [ 220.432334] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.342Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1174gitlab # [ 220.436112] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.345Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1175gitlab # [ 220.439955] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.350Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1176gitlab # [ 220.444921] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:12:58.354Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1177gitlab # [ 229.165543] gitlab-workhorse[816]: time="2026-10-02T14:13:07Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1178gitlab # [ 239.166519] gitlab-workhorse[816]: time="2026-10-02T14:13:17Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1179gitlab # [ 249.167797] gitlab-workhorse[816]: time="2026-10-02T14:13:27Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1180gitlab # [ 259.169187] gitlab-workhorse[816]: time="2026-10-02T14:13:37Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1181gitlab # [ 264.329187] postgres[3222]: [3222] ERROR: relation "table_without_models" does not exist1182gitlab # [ 264.329591] postgres[3222]: [3222] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1183gitlab # [ 265.855583] postgres[3242]: [3242] ERROR: relation "table_without_models" does not exist1184gitlab # [ 265.855827] postgres[3242]: [3242] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1185gitlab # [ 269.170214] gitlab-workhorse[816]: time="2026-10-02T14:13:47Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1186gitlab # [ 270.339645] bundle[2046]: {"timestamp":"2026-10-02T14:13:48.248Z","pid":2046,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1187gitlab # [ 270.340070] bundle[2046]: {"timestamp":"2026-10-02T14:13:48.249Z","pid":2046,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1188gitlab # [ 270.340230] bundle[2046]: {"timestamp":"2026-10-02T14:13:48.249Z","pid":2046,"message":"! #\u003cThread:0x0000f2236d597030 /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1189gitlab # [ 270.340308] bundle[2046]: {"timestamp":"2026-10-02T14:13:48.249Z","pid":2046,"message":"! #\u003cThread:0x0000f2236d7db490 /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1190gitlab # [ 270.340376] bundle[2046]: {"timestamp":"2026-10-02T14:13:48.249Z","pid":2046,"message":"Use Ctrl-C to stop"}1191gitlab # [ 270.340464] bundle[2046]: {"timestamp":"2026-10-02T14:13:48.249Z","pid":2046,"message":"Sending status to systemd every 1.0 sec"}1192gitlab # [ 271.135393] bundle[2046]: {"timestamp":"2026-10-02T14:13:49.044Z","pid":2046,"message":"- Worker 0 (PID: 3318) booted in 0.74s, phase: 0"}1193gitlab # [ 271.135722] bundle[2046]: {"timestamp":"2026-10-02T14:13:49.045Z","pid":2046,"message":"- Worker 1 (PID: 3324) booted in 0.71s, phase: 0"}1194gitlab # [ 271.148720] systemd[1]: Started gitlab.service.1195gitlab # [ 271.152341] systemd[1]: Reached target Common target for all GitLab services..1196gitlab # [ 271.158528] systemd[1]: Starting setup-gitlab.service...1197gitlab # [ 271.195546] setup-gitlab-start[3354]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1198gitlab # [ 271.276473] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:13:49.185Z","message":"Booted Rails 7.2.3.1 application in production environment"}1199gitlab # [ 271.279874] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:13:49.185Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1200gitlab # [ 271.284248] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:13:49.185Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1201gitlab # [ 271.287304] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:13:49.185Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1202gitlab # [ 271.290780] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:13:49.199Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1203gitlab # [ 274.330744] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:13:52.240Z","message":"Cleaning working queues"}1204gitlab # [ 274.331771] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:13:52.241Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"5b68c83b5172303565d11a1c","created_at":"2026-10-02T14:11:25.806Z","trace_propagation_headers":{"sentry-trace":"920fed9d08284e50b77d79a20604ee39-55c5166f957f4f79","baggage":"sentry-trace_id=920fed9d08284e50b77d79a20604ee39,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"29d577b5c3b0de2b1b7db38edb8358a3","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-02T14:11:25.821Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-5b68c83b5172303565d11a1c: start","job_status":"start","queue_duration_s":146.420133,"scheduling_latency_s":146.420194}1205gitlab # [ 274.584875] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:13:52.494Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"5b68c83b5172303565d11a1c","created_at":"2026-10-02T14:11:25.806Z","trace_propagation_headers":{"sentry-trace":"920fed9d08284e50b77d79a20604ee39-55c5166f957f4f79","baggage":"sentry-trace_id=920fed9d08284e50b77d79a20604ee39,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"29d577b5c3b0de2b1b7db38edb8358a3","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-02T14:11:25.821Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-5b68c83b5172303565d11a1c: done: 0.252857 sec","job_status":"done","queue_duration_s":146.420133,"scheduling_latency_s":146.420194,"redis_calls":24,"redis_duration_s":0.013334,"redis_read_bytes":912,"redis_write_bytes":2622,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.008751,"redis_feature_flag_read_bytes":902,"redis_feature_flag_write_bytes":1375,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002962,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001621,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":9,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.121345,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.13789944,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.252857,"completed_at":"2026-10-02T14:13:52.494Z","load_balancing_strategy":"primary","db_duration_s":0.075123,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1206gitlab # [ 276.122144] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:13:54 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 48891207gitlab # [ 276.127335] setup-gitlab-start[3354]: + TOKEN=nixbot-test-token-of-20-chars1208gitlab # [ 276.129486] setup-gitlab-start[3354]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n'1209gitlab # [ 276.146798] runuser[3392]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1210gitlab # [ 276.792540] setup-gitlab-start[3393]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1211gitlab # [ 276.792926] setup-gitlab-start[3393]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1212gitlab # [ 276.795448] setup-gitlab-start[3393]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1213gitlab # [ 277.908242] setup-gitlab-start[3393]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1214gitlab # [ 277.908539] setup-gitlab-start[3393]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1215gitlab # [ 277.912285] setup-gitlab-start[3393]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1216gitlab # [ 287.009295] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:04.917Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"12dff1e3cf77090d32b73d55","created_at":"2026-10-02T14:14:04.824Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7511f733603c828ab1b4792b207a0609","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:04.913Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-12dff1e3cf77090d32b73d55: start","job_status":"start","queue_duration_s":0.004057,"scheduling_latency_s":0.004106}1217gitlab # [ 287.122057] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:05.031Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1218gitlab # [ 287.158831] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:05.068Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"12dff1e3cf77090d32b73d55","created_at":"2026-10-02T14:14:04.824Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7511f733603c828ab1b4792b207a0609","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:04.913Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-12dff1e3cf77090d32b73d55: done: 0.150359 sec","job_status":"done","queue_duration_s":0.004057,"scheduling_latency_s":0.004106,"redis_calls":10,"redis_duration_s":0.012594,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000882,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.009859,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001853,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036266,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00395134,"duration_s":0.150359,"completed_at":"2026-10-02T14:14:05.068Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.04558,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1219gitlab # [ 287.224520] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:05.074Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"3513e69e02af8cbc0e87a66c","created_at":"2026-10-02T14:14:04.929Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0284ecd65d5c2b8523b60a1e4f3f0eae","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:04.958Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-3513e69e02af8cbc0e87a66c: start","job_status":"start","queue_duration_s":0.115777,"scheduling_latency_s":0.115821}1220gitlab # [ 287.271548] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:05.180Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"3513e69e02af8cbc0e87a66c","created_at":"2026-10-02T14:14:04.929Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0284ecd65d5c2b8523b60a1e4f3f0eae","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:04.958Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-3513e69e02af8cbc0e87a66c: done: 0.106408 sec","job_status":"done","queue_duration_s":0.115777,"scheduling_latency_s":0.115821,"redis_calls":12,"redis_duration_s":0.003979,"redis_read_bytes":11,"redis_write_bytes":1482,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001401,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00174,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000838,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.049,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.049,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031601,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00048158,"duration_s":0.106408,"completed_at":"2026-10-02T14:14:05.180Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.049441,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1221gitlab # [ 287.334967] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:05.183Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"ec802f616e7f824c8be5d192","created_at":"2026-10-02T14:14:04.972Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3b2341fb91833c15e7d2e8a2edeaaa9d","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:04.983Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-ec802f616e7f824c8be5d192: start","job_status":"start","queue_duration_s":0.200233,"scheduling_latency_s":0.200265}1222gitlab # [ 287.350192] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:05.248Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"ec802f616e7f824c8be5d192","created_at":"2026-10-02T14:14:04.972Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3b2341fb91833c15e7d2e8a2edeaaa9d","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:04.983Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-ec802f616e7f824c8be5d192: done: 0.065138 sec","job_status":"done","queue_duration_s":0.200233,"scheduling_latency_s":0.200265,"redis_calls":12,"redis_duration_s":0.00546,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.002849,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000644,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00141,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000557,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024115,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00046822,"duration_s":0.065138,"completed_at":"2026-10-02T14:14:05.248Z","load_balancing_strategy":"primary","db_duration_s":0.002852,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1223gitlab # [ 287.396337] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:05.255Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"f3241a081c77be1f0a97c497","created_at":"2026-10-02T14:14:05.013Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9863fa70daa3782b5cc2dbb25d759f8a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:05.024Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-f3241a081c77be1f0a97c497: start","job_status":"start","queue_duration_s":0.231091,"scheduling_latency_s":0.231123}1224gitlab # [ 287.717848] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:05.627Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"f3241a081c77be1f0a97c497","created_at":"2026-10-02T14:14:05.013Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9863fa70daa3782b5cc2dbb25d759f8a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:05.024Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-f3241a081c77be1f0a97c497: done: 0.37113 sec","job_status":"done","queue_duration_s":0.231091,"scheduling_latency_s":0.231123,"redis_calls":14,"redis_duration_s":0.009703,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002518,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004198,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002987,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.06,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.046,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.114417,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00102746,"exclusive_lock_hold_duration_s":0.25283126,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.37113,"completed_at":"2026-10-02T14:14:05.626Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.077264,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1225gitlab # [ 287.788639] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:05.633Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"05819cb5e23019bd9c7e8272","created_at":"2026-10-02T14:14:05.046Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"518b08e5e4d59741ea09d6407cf7b3ad","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-02T14:14:05.070Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-05819cb5e23019bd9c7e8272: start","job_status":"start","queue_duration_s":0.56297,"scheduling_latency_s":0.563016}1226gitlab # [ 287.805033] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:05.679Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1227gitlab # [ 287.809327] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:05.700Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"05819cb5e23019bd9c7e8272","created_at":"2026-10-02T14:14:05.046Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"518b08e5e4d59741ea09d6407cf7b3ad","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-02T14:14:05.070Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-05819cb5e23019bd9c7e8272: done: 0.067155 sec","job_status":"done","queue_duration_s":0.56297,"scheduling_latency_s":0.563016,"redis_calls":10,"redis_duration_s":0.003469,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001602,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001381,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000486,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021383,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00043772,"duration_s":0.067155,"completed_at":"2026-10-02T14:14:05.700Z","load_balancing_strategy":"primary","db_duration_s":0.019317,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1228gitlab # [ 287.854347] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:05.703Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"8fad47f503816391ffac8cae","created_at":"2026-10-02T14:14:05.151Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b0d65b8a83b7da92dc11cced5a0d83f7","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:05.176Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-8fad47f503816391ffac8cae: start","job_status":"start","queue_duration_s":0.526307,"scheduling_latency_s":0.526335}1229gitlab # [ 287.890361] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:05.799Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"8fad47f503816391ffac8cae","created_at":"2026-10-02T14:14:05.151Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b0d65b8a83b7da92dc11cced5a0d83f7","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:05.176Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-8fad47f503816391ffac8cae: done: 0.096338 sec","job_status":"done","queue_duration_s":0.526307,"scheduling_latency_s":0.526335,"redis_calls":10,"redis_duration_s":0.003435,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000585,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002166,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000684,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032627,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00050848,"duration_s":0.096338,"completed_at":"2026-10-02T14:14:05.799Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.024134,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1230gitlab # [ 287.942692] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:05.802Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"d33d73210712203676bf3e26","created_at":"2026-10-02T14:14:05.200Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:05.232Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-d33d73210712203676bf3e26: start","job_status":"start","queue_duration_s":0.569894,"scheduling_latency_s":0.569922}1231gitlab # [ 288.208759] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:06.117Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"d33d73210712203676bf3e26","created_at":"2026-10-02T14:14:05.200Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:05.232Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-d33d73210712203676bf3e26: done: 0.315107 sec","job_status":"done","queue_duration_s":0.569894,"scheduling_latency_s":0.569922,"redis_calls":71,"redis_duration_s":0.042076999999999996,"redis_read_bytes":120,"redis_write_bytes":42674,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002549,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.001216,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.037271,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001041,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.205189,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00101394,"duration_s":0.315107,"completed_at":"2026-10-02T14:14:06.117Z","load_balancing_strategy":"primary","db_duration_s":0.016528,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1232gitlab # [ 288.284556] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:06.120Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"5b8ba17c0e446e4b469114eb","created_at":"2026-10-02T14:14:05.257Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5b55aa7fbf1f66db391f48a7805c256e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-02T14:14:05.267Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-5b8ba17c0e446e4b469114eb: start","job_status":"start","queue_duration_s":0.852673,"scheduling_latency_s":0.852707}1233gitlab # [ 288.304460] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:06.142Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1234gitlab # [ 288.308878] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:06.156Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"5b8ba17c0e446e4b469114eb","created_at":"2026-10-02T14:14:05.257Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5b55aa7fbf1f66db391f48a7805c256e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-02T14:14:05.267Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-5b8ba17c0e446e4b469114eb: done: 0.036235 sec","job_status":"done","queue_duration_s":0.852673,"scheduling_latency_s":0.852707,"redis_calls":10,"redis_duration_s":0.003303,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00078,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001844,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000679,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020461,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0005111,"duration_s":0.036235,"completed_at":"2026-10-02T14:14:06.156Z","load_balancing_strategy":"primary","db_duration_s":0.000906,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1235gitlab # [ 288.356040] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:06.158Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"056c03feadfe559f1b207fd5","created_at":"2026-10-02T14:14:05.298Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5eea2129f9999270a2e4326ad93fb3b6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:05.312Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-056c03feadfe559f1b207fd5: start","job_status":"start","queue_duration_s":0.845443,"scheduling_latency_s":0.845474}1236gitlab # [ 288.570451] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:06.479Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"056c03feadfe559f1b207fd5","created_at":"2026-10-02T14:14:05.298Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5eea2129f9999270a2e4326ad93fb3b6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:05.312Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-056c03feadfe559f1b207fd5: done: 0.321658 sec","job_status":"done","queue_duration_s":0.845443,"scheduling_latency_s":0.845474,"redis_calls":14,"redis_duration_s":0.004981,"redis_read_bytes":13,"redis_write_bytes":1645,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001573,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001826,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001582,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.045,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.045,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.261981,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00093102,"exclusive_lock_hold_duration_s":0.27990332,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.321658,"completed_at":"2026-10-02T14:14:06.479Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.066312,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1237gitlab # [ 288.645343] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:06.482Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"a10d96ed9c7fee81e7c1622c","created_at":"2026-10-02T14:14:05.357Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7215c22245cb24ecfd9589f86a2447c4","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:05.371Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-a10d96ed9c7fee81e7c1622c: start","job_status":"start","queue_duration_s":1.110502,"scheduling_latency_s":1.11053}1238gitlab # [ 288.663316] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:06.528Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"a10d96ed9c7fee81e7c1622c","created_at":"2026-10-02T14:14:05.357Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7215c22245cb24ecfd9589f86a2447c4","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:05.371Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-a10d96ed9c7fee81e7c1622c: done: 0.046222 sec","job_status":"done","queue_duration_s":1.110502,"scheduling_latency_s":1.11053,"redis_calls":12,"redis_duration_s":0.004228,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.0007,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001885,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001643,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028746,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00138664,"exclusive_lock_hold_duration_s":0.011815619999999999,"duration_s":0.046222,"completed_at":"2026-10-02T14:14:06.528Z","load_balancing_strategy":"primary","db_duration_s":0.004972,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1239gitlab # [ 288.713486] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:06.531Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"2cdc7beae239fc531ae5516c","created_at":"2026-10-02T14:14:05.381Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5cb9c35b6a4a7dba9a9b931f309a8650","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:05.394Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-2cdc7beae239fc531ae5516c: start","job_status":"start","queue_duration_s":1.137084,"scheduling_latency_s":1.137114}1240gitlab # [ 288.730438] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:06.574Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"2cdc7beae239fc531ae5516c","created_at":"2026-10-02T14:14:05.381Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5cb9c35b6a4a7dba9a9b931f309a8650","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:05.394Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-2cdc7beae239fc531ae5516c: done: 0.043119 sec","job_status":"done","queue_duration_s":1.137084,"scheduling_latency_s":1.137114,"redis_calls":12,"redis_duration_s":0.004341,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000721,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001953,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001667,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025382,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00089138,"exclusive_lock_hold_duration_s":0.00840476,"duration_s":0.043119,"completed_at":"2026-10-02T14:14:06.574Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003953,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1241gitlab # [ 288.779862] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:06.576Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"612eba154b0d30f18ea8f20e","created_at":"2026-10-02T14:14:05.405Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8f0aa2730b70ecaa20ac96cd3460bbb1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:05.416Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-612eba154b0d30f18ea8f20e: start","job_status":"start","queue_duration_s":1.159914,"scheduling_latency_s":1.159943}1242gitlab # [ 288.796076] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:06.647Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"612eba154b0d30f18ea8f20e","created_at":"2026-10-02T14:14:05.405Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8f0aa2730b70ecaa20ac96cd3460bbb1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:05.416Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-612eba154b0d30f18ea8f20e: done: 0.070569 sec","job_status":"done","queue_duration_s":1.159914,"scheduling_latency_s":1.159943,"redis_calls":12,"redis_duration_s":0.005576,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000692,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002904,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00198,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03776,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00182346,"exclusive_lock_hold_duration_s":0.0185449,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.070569,"completed_at":"2026-10-02T14:14:06.647Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.020673,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1243gitlab # [ 288.843609] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:06.650Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"cc6c08d7c863dabcb74f1d00","created_at":"2026-10-02T14:14:05.428Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f64fba948ed32065b771c3739b6dfaea","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:05.439Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-cc6c08d7c863dabcb74f1d00: start","job_status":"start","queue_duration_s":1.210821,"scheduling_latency_s":1.210849}1244gitlab # [ 289.173527] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:07.082Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"cc6c08d7c863dabcb74f1d00","created_at":"2026-10-02T14:14:05.428Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f64fba948ed32065b771c3739b6dfaea","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:05.439Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-cc6c08d7c863dabcb74f1d00: done: 0.432292 sec","job_status":"done","queue_duration_s":1.210821,"scheduling_latency_s":1.210849,"redis_calls":565,"redis_duration_s":0.10514,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000705,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001767,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.102668,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.371724,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00046682,"duration_s":0.432292,"completed_at":"2026-10-02T14:14:07.082Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000779,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1245gitlab # [ 289.241934] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:07.085Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"0bb275dcb1f5a1de5252d298","created_at":"2026-10-02T14:14:05.445Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"77a4ec04337561eed83c2f540bc27750","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:05.456Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-0bb275dcb1f5a1de5252d298: start","job_status":"start","queue_duration_s":1.628148,"scheduling_latency_s":1.628177}1246gitlab # [ 289.259743] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:07.120Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"0bb275dcb1f5a1de5252d298","created_at":"2026-10-02T14:14:05.445Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"77a4ec04337561eed83c2f540bc27750","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:05.456Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-0bb275dcb1f5a1de5252d298: done: 0.035882 sec","job_status":"done","queue_duration_s":1.628148,"scheduling_latency_s":1.628177,"redis_calls":10,"redis_duration_s":0.0034409999999999996,"redis_read_bytes":9,"redis_write_bytes":1299,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000975,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001829,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000637,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019825,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0004709,"duration_s":0.035882,"completed_at":"2026-10-02T14:14:07.120Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004453,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1247gitlab # [ 289.307044] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:07.123Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"03108b85287ee0b85f982d8c","created_at":"2026-10-02T14:14:05.476Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2d63049d842a60310990221952bfa077","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:05.496Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-03108b85287ee0b85f982d8c: start","job_status":"start","queue_duration_s":1.626865,"scheduling_latency_s":1.626892}1248gitlab # [ 289.322799] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:07.142Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1249gitlab # [ 289.326328] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:07.154Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"03108b85287ee0b85f982d8c","created_at":"2026-10-02T14:14:05.476Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2d63049d842a60310990221952bfa077","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:05.496Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-03108b85287ee0b85f982d8c: done: 0.03162 sec","job_status":"done","queue_duration_s":1.626865,"scheduling_latency_s":1.626892,"redis_calls":10,"redis_duration_s":0.002803,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000573,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001558,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000672,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019459,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00046454,"duration_s":0.03162,"completed_at":"2026-10-02T14:14:07.154Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000724,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1250gitlab # [ 289.372344] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:07.157Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"f376ba7dea185f5e3b25556c","created_at":"2026-10-02T14:14:05.502Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"914c5af05ebf7989bd36d1acd27eb72b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:05.523Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-f376ba7dea185f5e3b25556c: start","job_status":"start","queue_duration_s":1.634081,"scheduling_latency_s":1.634112}1251gitlab # [ 289.388117] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:07.179Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1252gitlab # [ 289.392176] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:07.193Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"f376ba7dea185f5e3b25556c","created_at":"2026-10-02T14:14:05.502Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"914c5af05ebf7989bd36d1acd27eb72b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:05.523Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-f376ba7dea185f5e3b25556c: done: 0.035949 sec","job_status":"done","queue_duration_s":1.634081,"scheduling_latency_s":1.634112,"redis_calls":10,"redis_duration_s":0.0029969999999999997,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000677,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001669,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000651,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022096,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0004666,"duration_s":0.035949,"completed_at":"2026-10-02T14:14:07.193Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000687,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1253gitlab # [ 289.440790] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:07.195Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"ef148bee0f1776d32ec7e114","created_at":"2026-10-02T14:14:05.552Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"028ee490e015c4cf3344ec3367ce766e","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:05.566Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-ef148bee0f1776d32ec7e114: start","job_status":"start","queue_duration_s":1.628776,"scheduling_latency_s":1.628807}1254gitlab # [ 289.457672] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:07.227Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"ef148bee0f1776d32ec7e114","created_at":"2026-10-02T14:14:05.552Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"028ee490e015c4cf3344ec3367ce766e","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:05.566Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-ef148bee0f1776d32ec7e114: done: 0.032347 sec","job_status":"done","queue_duration_s":1.628776,"scheduling_latency_s":1.628807,"redis_calls":12,"redis_duration_s":0.002697,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00067,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001087,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00094,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01998,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00070698,"exclusive_lock_hold_duration_s":0.004844279999999999,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.032347,"completed_at":"2026-10-02T14:14:07.227Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002031,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1255gitlab # [ 289.509533] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:07.229Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"da97a974d1b7ba73f10e0c3a","created_at":"2026-10-02T14:14:05.621Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ecc777dd7873bf3a61b2cceffc655827","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:05.636Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-da97a974d1b7ba73f10e0c3a: start","job_status":"start","queue_duration_s":1.59309,"scheduling_latency_s":1.593112}1256gitlab # [ 289.526050] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:07.279Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"da97a974d1b7ba73f10e0c3a","created_at":"2026-10-02T14:14:05.621Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ecc777dd7873bf3a61b2cceffc655827","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:05.636Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-da97a974d1b7ba73f10e0c3a: done: 0.050751 sec","job_status":"done","queue_duration_s":1.59309,"scheduling_latency_s":1.593112,"redis_calls":10,"redis_duration_s":0.002037,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000395,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001206,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000436,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023454,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00078186,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.050751,"completed_at":"2026-10-02T14:14:07.279Z","load_balancing_strategy":"primary","db_duration_s":0.017135,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1257gitlab # [ 289.576220] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:07.281Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"bad71c9add6aeca84ebe785a","created_at":"2026-10-02T14:14:05.670Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f377669ab5b36dd7ddbde6a7112f0876","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:05.697Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-bad71c9add6aeca84ebe785a: start","job_status":"start","queue_duration_s":1.584735,"scheduling_latency_s":1.584759}1258gitlab # [ 289.592011] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:07.335Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"bad71c9add6aeca84ebe785a","created_at":"2026-10-02T14:14:05.670Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f377669ab5b36dd7ddbde6a7112f0876","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:05.697Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-bad71c9add6aeca84ebe785a: done: 0.053813 sec","job_status":"done","queue_duration_s":1.584735,"scheduling_latency_s":1.584759,"redis_calls":19,"redis_duration_s":0.004093,"redis_read_bytes":22,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00099,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000342,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.002105,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000656,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039941,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034436,"duration_s":0.053813,"completed_at":"2026-10-02T14:14:07.335Z","load_balancing_strategy":"primary","db_duration_s":0.003055,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1259gitlab # [ 289.641408] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:07.337Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"491e378a597afd847fa14a75","created_at":"2026-10-02T14:14:05.710Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"75cbaa94f80f5462171679bdc58e4dae","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:05.726Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-491e378a597afd847fa14a75: start","job_status":"start","queue_duration_s":1.610531,"scheduling_latency_s":1.61055}1260gitlab # [ 289.656632] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:07.392Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"491e378a597afd847fa14a75","created_at":"2026-10-02T14:14:05.710Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"75cbaa94f80f5462171679bdc58e4dae","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:05.726Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-491e378a597afd847fa14a75: done: 0.054997 sec","job_status":"done","queue_duration_s":1.610531,"scheduling_latency_s":1.61055,"redis_calls":16,"redis_duration_s":0.005101,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.003306,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00133,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000465,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0367,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00043874,"duration_s":0.054997,"completed_at":"2026-10-02T14:14:07.392Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007288,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1261gitlab # [ 289.701748] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:07.394Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"2726e17efaf51ee40f49f133","created_at":"2026-10-02T14:14:05.766Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"97c8436a2d247cd463fd02d4f9a84369","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:05.796Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-2726e17efaf51ee40f49f133: start","job_status":"start","queue_duration_s":1.597477,"scheduling_latency_s":1.597502}1262gitlab # [ 289.808591] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:07.717Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"2726e17efaf51ee40f49f133","created_at":"2026-10-02T14:14:05.766Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"97c8436a2d247cd463fd02d4f9a84369","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:05.796Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-2726e17efaf51ee40f49f133: done: 0.323735 sec","job_status":"done","queue_duration_s":1.597477,"scheduling_latency_s":1.597502,"redis_calls":12,"redis_duration_s":0.004384,"redis_read_bytes":11,"redis_write_bytes":1447,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000661,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":319,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001853,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":586,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00187,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":542,"db_count":101,"db_write_count":0,"db_cached_count":99,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":101,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":99,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":100,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":99,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.014,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.291826,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00089192,"exclusive_lock_hold_duration_s":0.28871964,"extra.ci_update_locked_unknown_artifacts_worker.removed_count":0,"extra.ci_update_locked_unknown_artifacts_worker.locked_count":0,"duration_s":0.323735,"completed_at":"2026-10-02T14:14:07.717Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.025205,"urgency":"throttled","target_duration_s":300,"target_scheduling_latency_s":null}1263gitlab # [ 289.876938] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:07.720Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"261fe38b247f1d7881e6a9a3","created_at":"2026-10-02T14:14:05.823Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"82e8df0c699660bf9f2c8f084ea34e04","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:05.854Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-261fe38b247f1d7881e6a9a3: start","job_status":"start","queue_duration_s":1.865952,"scheduling_latency_s":1.865988}1264gitlab # [ 289.894244] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:07.768Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"261fe38b247f1d7881e6a9a3","created_at":"2026-10-02T14:14:05.823Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"82e8df0c699660bf9f2c8f084ea34e04","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:05.854Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-261fe38b247f1d7881e6a9a3: done: 0.048442 sec","job_status":"done","queue_duration_s":1.865952,"scheduling_latency_s":1.865988,"redis_calls":10,"redis_duration_s":0.003036,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000601,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001771,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000664,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031093,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0004969,"duration_s":0.048442,"completed_at":"2026-10-02T14:14:07.768Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005531,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1265gitlab # [ 289.942461] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:07.771Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"75c7f7e52854f9071bd3787f","created_at":"2026-10-02T14:14:05.886Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a5c7cc38b6176e9dac10ab87381dc1c6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-02T14:14:05.915Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-75c7f7e52854f9071bd3787f: start","job_status":"start","queue_duration_s":1.855476,"scheduling_latency_s":1.855512}1266gitlab # [ 289.957398] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:07.819Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"75c7f7e52854f9071bd3787f","created_at":"2026-10-02T14:14:05.886Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a5c7cc38b6176e9dac10ab87381dc1c6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-02T14:14:05.915Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-75c7f7e52854f9071bd3787f: done: 0.047847 sec","job_status":"done","queue_duration_s":1.855476,"scheduling_latency_s":1.855512,"redis_calls":10,"redis_duration_s":0.002691,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000556,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001535,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0006,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031584,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00035344,"duration_s":0.047847,"completed_at":"2026-10-02T14:14:07.819Z","load_balancing_strategy":"primary","db_duration_s":0.005504,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1267gitlab # [ 290.001408] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:07.821Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"4b7004fe635281f3ab19e94d","created_at":"2026-10-02T14:14:05.927Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ee3461130d93762a0f81cd10f9654da8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:05.951Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-4b7004fe635281f3ab19e94d: start","job_status":"start","queue_duration_s":1.869639,"scheduling_latency_s":1.869667}1268gitlab # [ 290.017327] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:07.837Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1269gitlab # [ 290.021114] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:07.847Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"4b7004fe635281f3ab19e94d","created_at":"2026-10-02T14:14:05.927Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ee3461130d93762a0f81cd10f9654da8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:05.951Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-4b7004fe635281f3ab19e94d: done: 0.026681 sec","job_status":"done","queue_duration_s":1.869639,"scheduling_latency_s":1.869667,"redis_calls":10,"redis_duration_s":0.002222,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00049,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001275,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000457,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015328,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034076,"duration_s":0.026681,"completed_at":"2026-10-02T14:14:07.847Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000662,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1270gitlab # [ 290.069253] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:07.849Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"a2157e155703c4ef045d1e3c","created_at":"2026-10-02T14:14:05.968Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"33c7cf876dd294e912bfbd30a5495b4b","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:05.993Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-a2157e155703c4ef045d1e3c: start","job_status":"start","queue_duration_s":1.856264,"scheduling_latency_s":1.856295}1271gitlab # [ 290.086751] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:07.892Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"a2157e155703c4ef045d1e3c","created_at":"2026-10-02T14:14:05.968Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"33c7cf876dd294e912bfbd30a5495b4b","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:05.993Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-a2157e155703c4ef045d1e3c: done: 0.042533 sec","job_status":"done","queue_duration_s":1.856264,"scheduling_latency_s":1.856295,"redis_calls":13,"redis_duration_s":0.003287,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000483,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001591,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001213,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028761,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01239934,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.042533,"completed_at":"2026-10-02T14:14:07.892Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003434,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1272gitlab # [ 290.138851] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:07.894Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3b82ff256eca38241036263d","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3b82ff256eca38241036263d: start","job_status":"start","queue_duration_s":1.793745,"scheduling_latency_s":1.793772}1273gitlab # [ 290.152638] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:07.924Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3b82ff256eca38241036263d","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3b82ff256eca38241036263d: done: 0.030796 sec","job_status":"done","queue_duration_s":1.793745,"scheduling_latency_s":1.793772,"redis_calls":16,"redis_duration_s":0.003865,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000533,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001065,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002267,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019205,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034574,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.030796,"completed_at":"2026-10-02T14:14:07.924Z","load_balancing_strategy":"primary","db_duration_s":0.000614,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1274gitlab # [ 290.196897] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:07.926Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c128bf3943de7fcd1e13bb89","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c128bf3943de7fcd1e13bb89: start","job_status":"start","queue_duration_s":1.826337,"scheduling_latency_s":1.826366}1275gitlab # [ 290.210344] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:07.949Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c128bf3943de7fcd1e13bb89","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c128bf3943de7fcd1e13bb89: done: 0.022229 sec","job_status":"done","queue_duration_s":1.826337,"scheduling_latency_s":1.826366,"redis_calls":13,"redis_duration_s":0.003182,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001022,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00216,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014489,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022229,"completed_at":"2026-10-02T14:14:07.948Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1276gitlab # [ 290.254597] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:07.950Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"43ad99590186e7394e7916c8","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-43ad99590186e7394e7916c8: start","job_status":"start","queue_duration_s":1.85047,"scheduling_latency_s":1.850499}1277gitlab # [ 290.268418] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:07.974Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"43ad99590186e7394e7916c8","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-43ad99590186e7394e7916c8: done: 0.023759 sec","job_status":"done","queue_duration_s":1.85047,"scheduling_latency_s":1.850499,"redis_calls":13,"redis_duration_s":0.003702,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001144,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002558,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014632,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023759,"completed_at":"2026-10-02T14:14:07.974Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1278gitlab # [ 290.312174] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:07.976Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e46d8142bc8882db967b0924","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e46d8142bc8882db967b0924: start","job_status":"start","queue_duration_s":1.876191,"scheduling_latency_s":1.876221}1279gitlab # [ 290.326573] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.012Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e46d8142bc8882db967b0924","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e46d8142bc8882db967b0924: done: 0.035619 sec","job_status":"done","queue_duration_s":1.876191,"scheduling_latency_s":1.876221,"redis_calls":13,"redis_duration_s":0.011618,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.006824,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004794,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018519,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.035619,"completed_at":"2026-10-02T14:14:08.012Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1280gitlab # [ 290.369158] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.014Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bd87b97ec87b2a995e784b97","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bd87b97ec87b2a995e784b97: start","job_status":"start","queue_duration_s":1.913747,"scheduling_latency_s":1.913778}1281gitlab # [ 290.382610] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.037Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bd87b97ec87b2a995e784b97","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bd87b97ec87b2a995e784b97: done: 0.023717 sec","job_status":"done","queue_duration_s":1.913747,"scheduling_latency_s":1.913778,"redis_calls":13,"redis_duration_s":0.003267,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001083,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002184,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015368,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023717,"completed_at":"2026-10-02T14:14:08.037Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1282gitlab # [ 290.425098] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.040Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"770fe1e2a881df743374380a","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-770fe1e2a881df743374380a: start","job_status":"start","queue_duration_s":1.939849,"scheduling_latency_s":1.939882}1283gitlab # [ 290.438455] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.064Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"770fe1e2a881df743374380a","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-770fe1e2a881df743374380a: done: 0.023945 sec","job_status":"done","queue_duration_s":1.939849,"scheduling_latency_s":1.939882,"redis_calls":13,"redis_duration_s":0.003723,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000931,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002792,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014891,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023945,"completed_at":"2026-10-02T14:14:08.064Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1284gitlab # [ 290.480794] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.066Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"24b9ebd8aba19ac1eeaaf38f","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-24b9ebd8aba19ac1eeaaf38f: start","job_status":"start","queue_duration_s":1.965913,"scheduling_latency_s":1.965945}1285gitlab # [ 290.493809] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.089Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"24b9ebd8aba19ac1eeaaf38f","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-24b9ebd8aba19ac1eeaaf38f: done: 0.022963 sec","job_status":"done","queue_duration_s":1.965913,"scheduling_latency_s":1.965945,"redis_calls":13,"redis_duration_s":0.002579,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000969,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00161,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015702,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022963,"completed_at":"2026-10-02T14:14:08.089Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1286gitlab # [ 290.535829] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.091Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"169030755e12abd073099bd2","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-169030755e12abd073099bd2: start","job_status":"start","queue_duration_s":1.991041,"scheduling_latency_s":1.991076}1287gitlab # [ 290.549153] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.115Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"169030755e12abd073099bd2","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-169030755e12abd073099bd2: done: 0.023525 sec","job_status":"done","queue_duration_s":1.991041,"scheduling_latency_s":1.991076,"redis_calls":13,"redis_duration_s":0.0029300000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001286,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001644,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0159,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023525,"completed_at":"2026-10-02T14:14:08.114Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1288gitlab # [ 290.590059] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.117Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9aace221788b519f5ff92f6a","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9aace221788b519f5ff92f6a: start","job_status":"start","queue_duration_s":2.016693,"scheduling_latency_s":2.016726}1289gitlab # [ 290.603116] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.140Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9aace221788b519f5ff92f6a","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9aace221788b519f5ff92f6a: done: 0.023812 sec","job_status":"done","queue_duration_s":2.016693,"scheduling_latency_s":2.016726,"redis_calls":13,"redis_duration_s":0.002711,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001002,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001709,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01616,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023812,"completed_at":"2026-10-02T14:14:08.140Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1290gitlab # [ 290.644060] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.143Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0ee4453b06ecf411a07dab30","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0ee4453b06ecf411a07dab30: start","job_status":"start","queue_duration_s":2.042611,"scheduling_latency_s":2.042645}1291gitlab # [ 290.656982] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.166Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0ee4453b06ecf411a07dab30","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0ee4453b06ecf411a07dab30: done: 0.023348 sec","job_status":"done","queue_duration_s":2.042611,"scheduling_latency_s":2.042645,"redis_calls":13,"redis_duration_s":0.002876,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00109,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001786,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015935,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023348,"completed_at":"2026-10-02T14:14:08.166Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1292gitlab # [ 290.697613] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.168Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3c60c9da3eec4b1b9ca82e7e","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3c60c9da3eec4b1b9ca82e7e: start","job_status":"start","queue_duration_s":2.067845,"scheduling_latency_s":2.067876}1293gitlab # [ 290.710554] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.190Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3c60c9da3eec4b1b9ca82e7e","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3c60c9da3eec4b1b9ca82e7e: done: 0.022431 sec","job_status":"done","queue_duration_s":2.067845,"scheduling_latency_s":2.067876,"redis_calls":13,"redis_duration_s":0.0025210000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000893,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001628,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014952,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022431,"completed_at":"2026-10-02T14:14:08.190Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1294gitlab # [ 290.752712] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.192Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e5ef1102265795752aaaf3de","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e5ef1102265795752aaaf3de: start","job_status":"start","queue_duration_s":2.092027,"scheduling_latency_s":2.092056}1295gitlab # [ 290.765600] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.215Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e5ef1102265795752aaaf3de","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e5ef1102265795752aaaf3de: done: 0.023151 sec","job_status":"done","queue_duration_s":2.092027,"scheduling_latency_s":2.092056,"redis_calls":13,"redis_duration_s":0.002497,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000894,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001603,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015488,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023151,"completed_at":"2026-10-02T14:14:08.215Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1296gitlab # [ 290.806462] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.217Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6d7bbff584ac7350fbcf5bf","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6d7bbff584ac7350fbcf5bf: start","job_status":"start","queue_duration_s":2.117188,"scheduling_latency_s":2.117219}1297gitlab # [ 290.819606] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.240Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6d7bbff584ac7350fbcf5bf","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6d7bbff584ac7350fbcf5bf: done: 0.022834 sec","job_status":"done","queue_duration_s":2.117188,"scheduling_latency_s":2.117219,"redis_calls":13,"redis_duration_s":0.002615,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001033,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001582,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014965,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022834,"completed_at":"2026-10-02T14:14:08.240Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1298gitlab # [ 290.860915] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.242Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6983eac58fb81f620c1632a","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6983eac58fb81f620c1632a: start","job_status":"start","queue_duration_s":2.14178,"scheduling_latency_s":2.14181}1299gitlab # [ 290.873790] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.265Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6983eac58fb81f620c1632a","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6983eac58fb81f620c1632a: done: 0.023104 sec","job_status":"done","queue_duration_s":2.14178,"scheduling_latency_s":2.14181,"redis_calls":13,"redis_duration_s":0.002578,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000993,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001585,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015113,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023104,"completed_at":"2026-10-02T14:14:08.265Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1300gitlab # [ 290.914668] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.267Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cefc8551835390dbe7d9c75b","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cefc8551835390dbe7d9c75b: start","job_status":"start","queue_duration_s":2.166736,"scheduling_latency_s":2.166767}1301gitlab # [ 290.927678] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.289Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cefc8551835390dbe7d9c75b","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cefc8551835390dbe7d9c75b: done: 0.022247 sec","job_status":"done","queue_duration_s":2.166736,"scheduling_latency_s":2.166767,"redis_calls":13,"redis_duration_s":0.002587,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001018,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001569,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01482,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022247,"completed_at":"2026-10-02T14:14:08.289Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1302gitlab # [ 290.969446] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.291Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d2b1df9c61997678744486c","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d2b1df9c61997678744486c: start","job_status":"start","queue_duration_s":2.191151,"scheduling_latency_s":2.191181}1303gitlab # [ 290.983145] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.315Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d2b1df9c61997678744486c","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d2b1df9c61997678744486c: done: 0.023996 sec","job_status":"done","queue_duration_s":2.191151,"scheduling_latency_s":2.191181,"redis_calls":13,"redis_duration_s":0.003178,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001504,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001674,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015184,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023996,"completed_at":"2026-10-02T14:14:08.315Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1304gitlab # [ 291.024198] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.317Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1c98741a59a9d55324ae2659","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1c98741a59a9d55324ae2659: start","job_status":"start","queue_duration_s":2.21674,"scheduling_latency_s":2.216766}1305gitlab # [ 291.037117] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.333Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1c98741a59a9d55324ae2659","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1c98741a59a9d55324ae2659: done: 0.0165 sec","job_status":"done","queue_duration_s":2.21674,"scheduling_latency_s":2.216766,"redis_calls":13,"redis_duration_s":0.0016849999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000601,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001084,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011418,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0165,"completed_at":"2026-10-02T14:14:08.333Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1306gitlab # [ 291.077755] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.335Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ecfc00be18ec6a43eab00b57","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ecfc00be18ec6a43eab00b57: start","job_status":"start","queue_duration_s":2.234807,"scheduling_latency_s":2.234825}1307gitlab # [ 291.090557] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.351Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ecfc00be18ec6a43eab00b57","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ecfc00be18ec6a43eab00b57: done: 0.016375 sec","job_status":"done","queue_duration_s":2.234807,"scheduling_latency_s":2.234825,"redis_calls":13,"redis_duration_s":0.002088,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001028,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00106,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011647,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016375,"completed_at":"2026-10-02T14:14:08.351Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1308gitlab # [ 291.131541] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.353Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c17cde21a2ec6e395f7ce3ce","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c17cde21a2ec6e395f7ce3ce: start","job_status":"start","queue_duration_s":2.252551,"scheduling_latency_s":2.252568}1309gitlab # [ 291.145885] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.369Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c17cde21a2ec6e395f7ce3ce","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c17cde21a2ec6e395f7ce3ce: done: 0.016458 sec","job_status":"done","queue_duration_s":2.252551,"scheduling_latency_s":2.252568,"redis_calls":13,"redis_duration_s":0.001905,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000589,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001316,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011376,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016458,"completed_at":"2026-10-02T14:14:08.369Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1310gitlab # [ 291.187617] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.370Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4ca30cffd240d0855e1b5e2b","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4ca30cffd240d0855e1b5e2b: start","job_status":"start","queue_duration_s":2.270465,"scheduling_latency_s":2.270483}1311gitlab # [ 291.200850] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.391Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4ca30cffd240d0855e1b5e2b","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4ca30cffd240d0855e1b5e2b: done: 0.020864 sec","job_status":"done","queue_duration_s":2.270465,"scheduling_latency_s":2.270483,"redis_calls":13,"redis_duration_s":0.00178,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000623,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001157,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011829,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020864,"completed_at":"2026-10-02T14:14:08.391Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1312gitlab # [ 291.241414] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.393Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"daa0f60d1c8377bb422725ba","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-daa0f60d1c8377bb422725ba: start","job_status":"start","queue_duration_s":2.292772,"scheduling_latency_s":2.29279}1313gitlab # [ 291.254601] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.410Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"daa0f60d1c8377bb422725ba","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-daa0f60d1c8377bb422725ba: done: 0.017648 sec","job_status":"done","queue_duration_s":2.292772,"scheduling_latency_s":2.29279,"redis_calls":13,"redis_duration_s":0.001983,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000686,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001297,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012321,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017648,"completed_at":"2026-10-02T14:14:08.410Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1314gitlab # [ 291.295294] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.412Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"29a10fe788b7a28079a2c5ab","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-29a10fe788b7a28079a2c5ab: start","job_status":"start","queue_duration_s":2.311997,"scheduling_latency_s":2.312016}1315gitlab # [ 291.308370] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.430Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"29a10fe788b7a28079a2c5ab","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-29a10fe788b7a28079a2c5ab: done: 0.018048 sec","job_status":"done","queue_duration_s":2.311997,"scheduling_latency_s":2.312016,"redis_calls":13,"redis_duration_s":0.0020629999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000707,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001356,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012454,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018048,"completed_at":"2026-10-02T14:14:08.430Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1316gitlab # [ 291.350102] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.432Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6078813df5a1bc952173b15f","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6078813df5a1bc952173b15f: start","job_status":"start","queue_duration_s":2.331702,"scheduling_latency_s":2.331721}1317gitlab # [ 291.363086] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.450Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6078813df5a1bc952173b15f","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6078813df5a1bc952173b15f: done: 0.017911 sec","job_status":"done","queue_duration_s":2.331702,"scheduling_latency_s":2.331721,"redis_calls":13,"redis_duration_s":0.002105,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000738,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001367,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012634,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017911,"completed_at":"2026-10-02T14:14:08.450Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1318gitlab # [ 291.403872] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.451Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0e6ced78d2158b698c8591f4","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0e6ced78d2158b698c8591f4: start","job_status":"start","queue_duration_s":2.351331,"scheduling_latency_s":2.35135}1319gitlab # [ 291.416980] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.470Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0e6ced78d2158b698c8591f4","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0e6ced78d2158b698c8591f4: done: 0.018342 sec","job_status":"done","queue_duration_s":2.351331,"scheduling_latency_s":2.35135,"redis_calls":13,"redis_duration_s":0.002201,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000762,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001439,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013055,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018342,"completed_at":"2026-10-02T14:14:08.470Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1320gitlab # [ 291.457428] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.471Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b236bfdd8b5ac230da60a930","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b236bfdd8b5ac230da60a930: start","job_status":"start","queue_duration_s":2.371059,"scheduling_latency_s":2.371077}1321gitlab # [ 291.470294] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.490Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b236bfdd8b5ac230da60a930","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b236bfdd8b5ac230da60a930: done: 0.019008 sec","job_status":"done","queue_duration_s":2.371059,"scheduling_latency_s":2.371077,"redis_calls":13,"redis_duration_s":0.00226,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000758,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001502,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012811,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019008,"completed_at":"2026-10-02T14:14:08.490Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1322gitlab # [ 291.510967] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.491Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f61e22e177dacb31acd87d2","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f61e22e177dacb31acd87d2: start","job_status":"start","queue_duration_s":2.39144,"scheduling_latency_s":2.391459}1323gitlab # [ 291.523891] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.510Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f61e22e177dacb31acd87d2","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f61e22e177dacb31acd87d2: done: 0.01906 sec","job_status":"done","queue_duration_s":2.39144,"scheduling_latency_s":2.391459,"redis_calls":13,"redis_duration_s":0.002756,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000755,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002001,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012918,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01906,"completed_at":"2026-10-02T14:14:08.510Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1324gitlab # [ 291.565608] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.512Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"78ece9a02a473e39adc0cb22","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-78ece9a02a473e39adc0cb22: start","job_status":"start","queue_duration_s":2.411875,"scheduling_latency_s":2.411895}1325gitlab # [ 291.578515] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.531Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"78ece9a02a473e39adc0cb22","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-78ece9a02a473e39adc0cb22: done: 0.019018 sec","job_status":"done","queue_duration_s":2.411875,"scheduling_latency_s":2.411895,"redis_calls":13,"redis_duration_s":0.002304,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000797,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001507,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012659,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019018,"completed_at":"2026-10-02T14:14:08.531Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1326gitlab # [ 291.619257] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.532Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"deb70d15c1878aae8c383528","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-deb70d15c1878aae8c383528: start","job_status":"start","queue_duration_s":2.432317,"scheduling_latency_s":2.432336}1327gitlab # [ 291.632258] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.552Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"deb70d15c1878aae8c383528","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-deb70d15c1878aae8c383528: done: 0.019565 sec","job_status":"done","queue_duration_s":2.432317,"scheduling_latency_s":2.432336,"redis_calls":13,"redis_duration_s":0.0023810000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000863,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001518,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013118,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019565,"completed_at":"2026-10-02T14:14:08.552Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1328gitlab # [ 291.673138] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.554Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4bcceee93aae85d5c99c9011","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4bcceee93aae85d5c99c9011: start","job_status":"start","queue_duration_s":2.453671,"scheduling_latency_s":2.45369}1329gitlab # [ 291.685989] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.573Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4bcceee93aae85d5c99c9011","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4bcceee93aae85d5c99c9011: done: 0.019554 sec","job_status":"done","queue_duration_s":2.453671,"scheduling_latency_s":2.45369,"redis_calls":13,"redis_duration_s":0.002386,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000881,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001505,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013284,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019554,"completed_at":"2026-10-02T14:14:08.573Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1330gitlab # [ 291.726829] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.575Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"10b74b79fee5f25316f8d47e","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-10b74b79fee5f25316f8d47e: start","job_status":"start","queue_duration_s":2.475234,"scheduling_latency_s":2.475253}1331gitlab # [ 291.739847] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.595Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"10b74b79fee5f25316f8d47e","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-10b74b79fee5f25316f8d47e: done: 0.019487 sec","job_status":"done","queue_duration_s":2.475234,"scheduling_latency_s":2.475253,"redis_calls":13,"redis_duration_s":0.002443,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00087,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001573,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013331,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019487,"completed_at":"2026-10-02T14:14:08.595Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1332gitlab # [ 291.781426] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.597Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e98649dc92b6293dfee8fa83","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e98649dc92b6293dfee8fa83: start","job_status":"start","queue_duration_s":2.496528,"scheduling_latency_s":2.496547}1333gitlab # [ 291.794340] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.616Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e98649dc92b6293dfee8fa83","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e98649dc92b6293dfee8fa83: done: 0.019731 sec","job_status":"done","queue_duration_s":2.496528,"scheduling_latency_s":2.496547,"redis_calls":13,"redis_duration_s":0.00272,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000889,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001831,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013773,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019731,"completed_at":"2026-10-02T14:14:08.616Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1334gitlab # [ 291.835013] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.618Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"045c0bfbbb56bd6e6794954d","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-045c0bfbbb56bd6e6794954d: start","job_status":"start","queue_duration_s":2.518137,"scheduling_latency_s":2.518156}1335gitlab # [ 291.848070] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.638Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"045c0bfbbb56bd6e6794954d","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-045c0bfbbb56bd6e6794954d: done: 0.019704 sec","job_status":"done","queue_duration_s":2.518137,"scheduling_latency_s":2.518156,"redis_calls":13,"redis_duration_s":0.002458,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000858,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0016,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013603,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019704,"completed_at":"2026-10-02T14:14:08.638Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1336gitlab # [ 291.891992] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.640Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d97af6a9e7b29e9965f2e6a5","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d97af6a9e7b29e9965f2e6a5: start","job_status":"start","queue_duration_s":2.539897,"scheduling_latency_s":2.539917}1337gitlab # [ 291.906021] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.658Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d97af6a9e7b29e9965f2e6a5","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d97af6a9e7b29e9965f2e6a5: done: 0.01779 sec","job_status":"done","queue_duration_s":2.539897,"scheduling_latency_s":2.539917,"redis_calls":13,"redis_duration_s":0.0019950000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000714,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001281,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012309,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01779,"completed_at":"2026-10-02T14:14:08.658Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1338gitlab # [ 291.950539] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.659Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98e061bb6c4b83e6b0271d8c","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98e061bb6c4b83e6b0271d8c: start","job_status":"start","queue_duration_s":2.559327,"scheduling_latency_s":2.559346}1339gitlab # [ 291.965199] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.677Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98e061bb6c4b83e6b0271d8c","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98e061bb6c4b83e6b0271d8c: done: 0.017276 sec","job_status":"done","queue_duration_s":2.559327,"scheduling_latency_s":2.559346,"redis_calls":13,"redis_duration_s":0.001923,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000676,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001247,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012197,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017276,"completed_at":"2026-10-02T14:14:08.677Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1340gitlab # [ 292.010338] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.678Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f6df893db455da2492845856","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f6df893db455da2492845856: start","job_status":"start","queue_duration_s":2.577901,"scheduling_latency_s":2.577922}1341gitlab # [ 292.023776] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.696Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f6df893db455da2492845856","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f6df893db455da2492845856: done: 0.017986 sec","job_status":"done","queue_duration_s":2.577901,"scheduling_latency_s":2.577922,"redis_calls":13,"redis_duration_s":0.002118,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000658,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00146,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012116,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017986,"completed_at":"2026-10-02T14:14:08.696Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1342gitlab # [ 292.066088] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.697Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e7b22e28dc6be9a41e68c8ba","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e7b22e28dc6be9a41e68c8ba: start","job_status":"start","queue_duration_s":2.597152,"scheduling_latency_s":2.59717}1343gitlab # [ 292.079478] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.715Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e7b22e28dc6be9a41e68c8ba","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e7b22e28dc6be9a41e68c8ba: done: 0.018155 sec","job_status":"done","queue_duration_s":2.597152,"scheduling_latency_s":2.59717,"redis_calls":13,"redis_duration_s":0.002267,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000721,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001546,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012375,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018155,"completed_at":"2026-10-02T14:14:08.715Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1344gitlab # [ 292.121987] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.717Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"998070c1093fbdd97bef6628","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-998070c1093fbdd97bef6628: start","job_status":"start","queue_duration_s":2.61653,"scheduling_latency_s":2.616547}1345gitlab # [ 292.135439] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.735Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"998070c1093fbdd97bef6628","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-998070c1093fbdd97bef6628: done: 0.018283 sec","job_status":"done","queue_duration_s":2.61653,"scheduling_latency_s":2.616547,"redis_calls":13,"redis_duration_s":0.0026049999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001019,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001586,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012268,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018283,"completed_at":"2026-10-02T14:14:08.735Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1346gitlab # [ 292.178250] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.736Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"54a5fbd1c6a6bca2119c7163","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-54a5fbd1c6a6bca2119c7163: start","job_status":"start","queue_duration_s":2.636125,"scheduling_latency_s":2.636145}1347gitlab # [ 292.191309] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.754Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"54a5fbd1c6a6bca2119c7163","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-54a5fbd1c6a6bca2119c7163: done: 0.018068 sec","job_status":"done","queue_duration_s":2.636125,"scheduling_latency_s":2.636145,"redis_calls":13,"redis_duration_s":0.002018,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000725,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001293,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01221,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018068,"completed_at":"2026-10-02T14:14:08.754Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1348gitlab # [ 292.232106] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.756Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"555cda269bbf7c8f2ecd2c64","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-555cda269bbf7c8f2ecd2c64: start","job_status":"start","queue_duration_s":2.655647,"scheduling_latency_s":2.655665}1349gitlab # [ 292.244855] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.772Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"555cda269bbf7c8f2ecd2c64","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-555cda269bbf7c8f2ecd2c64: done: 0.016105 sec","job_status":"done","queue_duration_s":2.655647,"scheduling_latency_s":2.655665,"redis_calls":13,"redis_duration_s":0.001639,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000598,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001041,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011029,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016105,"completed_at":"2026-10-02T14:14:08.772Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1350gitlab # [ 292.285606] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.773Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0446d0f7d8554f9686620d42","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0446d0f7d8554f9686620d42: start","job_status":"start","queue_duration_s":2.673145,"scheduling_latency_s":2.673163}1351gitlab # [ 292.298933] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.789Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0446d0f7d8554f9686620d42","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0446d0f7d8554f9686620d42: done: 0.015859 sec","job_status":"done","queue_duration_s":2.673145,"scheduling_latency_s":2.673163,"redis_calls":13,"redis_duration_s":0.001794,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000614,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00118,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010941,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015859,"completed_at":"2026-10-02T14:14:08.789Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1352gitlab # [ 292.340834] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.790Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d90aae6a0151ae7438d08dc7","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d90aae6a0151ae7438d08dc7: start","job_status":"start","queue_duration_s":2.690381,"scheduling_latency_s":2.690398}1353gitlab # [ 292.354069] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.806Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d90aae6a0151ae7438d08dc7","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d90aae6a0151ae7438d08dc7: done: 0.016135 sec","job_status":"done","queue_duration_s":2.690381,"scheduling_latency_s":2.690398,"redis_calls":13,"redis_duration_s":0.001924,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000589,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001335,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011238,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016135,"completed_at":"2026-10-02T14:14:08.806Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1354gitlab # [ 292.397105] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.808Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"713c3fca9cec1b1c9b618c3b","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-713c3fca9cec1b1c9b618c3b: start","job_status":"start","queue_duration_s":2.708083,"scheduling_latency_s":2.708102}1355gitlab # [ 292.410523] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.824Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"713c3fca9cec1b1c9b618c3b","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-713c3fca9cec1b1c9b618c3b: done: 0.01615 sec","job_status":"done","queue_duration_s":2.708083,"scheduling_latency_s":2.708102,"redis_calls":13,"redis_duration_s":0.001657,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000581,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001076,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011479,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01615,"completed_at":"2026-10-02T14:14:08.824Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1356gitlab # [ 292.452587] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.826Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0ccb7d69f38668ac37f4e24e","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0ccb7d69f38668ac37f4e24e: start","job_status":"start","queue_duration_s":2.725599,"scheduling_latency_s":2.725617}1357gitlab # [ 292.465888] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.842Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0ccb7d69f38668ac37f4e24e","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0ccb7d69f38668ac37f4e24e: done: 0.01596 sec","job_status":"done","queue_duration_s":2.725599,"scheduling_latency_s":2.725617,"redis_calls":13,"redis_duration_s":0.0016940000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000599,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001095,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011327,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01596,"completed_at":"2026-10-02T14:14:08.841Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1358gitlab # [ 292.508148] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.843Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"755e9428661b8045eb68ed9c","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-755e9428661b8045eb68ed9c: start","job_status":"start","queue_duration_s":2.743098,"scheduling_latency_s":2.743116}1359gitlab # [ 292.521421] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.859Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"755e9428661b8045eb68ed9c","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-755e9428661b8045eb68ed9c: done: 0.016389 sec","job_status":"done","queue_duration_s":2.743098,"scheduling_latency_s":2.743116,"redis_calls":13,"redis_duration_s":0.001781,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000642,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001139,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011632,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016389,"completed_at":"2026-10-02T14:14:08.859Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1360gitlab # [ 292.563332] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.861Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6dcbf266514718cce8b6e692","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6dcbf266514718cce8b6e692: start","job_status":"start","queue_duration_s":2.760643,"scheduling_latency_s":2.76066}1361gitlab # [ 292.577685] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.877Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6dcbf266514718cce8b6e692","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6dcbf266514718cce8b6e692: done: 0.016761 sec","job_status":"done","queue_duration_s":2.760643,"scheduling_latency_s":2.76066,"redis_calls":13,"redis_duration_s":0.001839,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000648,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001191,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011386,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016761,"completed_at":"2026-10-02T14:14:08.877Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1362gitlab # [ 292.619801] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.879Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8043f056840cc4af0768abd3","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8043f056840cc4af0768abd3: start","job_status":"start","queue_duration_s":2.778709,"scheduling_latency_s":2.778728}1363gitlab # [ 292.633311] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.896Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8043f056840cc4af0768abd3","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8043f056840cc4af0768abd3: done: 0.016971 sec","job_status":"done","queue_duration_s":2.778709,"scheduling_latency_s":2.778728,"redis_calls":13,"redis_duration_s":0.001839,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000651,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001188,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011782,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016971,"completed_at":"2026-10-02T14:14:08.896Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1364gitlab # [ 292.675325] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.897Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"53f1413f7561f853165703ea","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-53f1413f7561f853165703ea: start","job_status":"start","queue_duration_s":2.796881,"scheduling_latency_s":2.796898}1365gitlab # [ 292.688823] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.915Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"53f1413f7561f853165703ea","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-53f1413f7561f853165703ea: done: 0.018633 sec","job_status":"done","queue_duration_s":2.796881,"scheduling_latency_s":2.796898,"redis_calls":13,"redis_duration_s":0.0023499999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001034,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001316,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012041,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018633,"completed_at":"2026-10-02T14:14:08.915Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1366gitlab # [ 292.731963] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.917Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f62c9feed67a37ef9499b85b","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f62c9feed67a37ef9499b85b: start","job_status":"start","queue_duration_s":2.816775,"scheduling_latency_s":2.816793}1367gitlab # [ 292.746033] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.936Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f62c9feed67a37ef9499b85b","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f62c9feed67a37ef9499b85b: done: 0.019061 sec","job_status":"done","queue_duration_s":2.816775,"scheduling_latency_s":2.816793,"redis_calls":13,"redis_duration_s":0.002091,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000738,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001353,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012638,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019061,"completed_at":"2026-10-02T14:14:08.936Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1368gitlab # [ 292.790774] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.937Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"db55f23abba9867ae6bde769","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-db55f23abba9867ae6bde769: start","job_status":"start","queue_duration_s":2.837159,"scheduling_latency_s":2.837177}1369gitlab # [ 292.804784] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.957Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"db55f23abba9867ae6bde769","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-db55f23abba9867ae6bde769: done: 0.019813 sec","job_status":"done","queue_duration_s":2.837159,"scheduling_latency_s":2.837177,"redis_calls":13,"redis_duration_s":0.002483,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000906,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001577,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013086,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019813,"completed_at":"2026-10-02T14:14:08.957Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1370gitlab # [ 292.847225] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.959Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3ca749aaa8bfc5523c1404f1","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3ca749aaa8bfc5523c1404f1: start","job_status":"start","queue_duration_s":2.858792,"scheduling_latency_s":2.858811}1371gitlab # [ 292.867597] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.978Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3ca749aaa8bfc5523c1404f1","created_at":"2026-10-02T14:14:05.852Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-9558c050530140cf","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac4983939412e6d97390f6d142ee37a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:06.100Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3ca749aaa8bfc5523c1404f1: done: 0.019247 sec","job_status":"done","queue_duration_s":2.858792,"scheduling_latency_s":2.858811,"redis_calls":13,"redis_duration_s":0.002394,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000818,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001576,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013416,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019247,"completed_at":"2026-10-02T14:14:08.978Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1372gitlab # [ 292.926308] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:08.980Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b0ea547589d23380eac6520f","created_at":"2026-10-02T14:14:07.305Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-e0182bb6f1954479","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f377669ab5b36dd7ddbde6a7112f0876","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:07.325Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b0ea547589d23380eac6520f: start","job_status":"start","queue_duration_s":1.654971,"scheduling_latency_s":1.65499}1373gitlab # [ 292.940933] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:09.007Z","message":"No existing merge request to be cleaned up."}1374gitlab # [ 292.942771] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:09.026Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b0ea547589d23380eac6520f","created_at":"2026-10-02T14:14:07.305Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-e0182bb6f1954479","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f377669ab5b36dd7ddbde6a7112f0876","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:07.325Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b0ea547589d23380eac6520f: done: 0.046251 sec","job_status":"done","queue_duration_s":1.654971,"scheduling_latency_s":1.65499,"redis_calls":13,"redis_duration_s":0.002892,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000476,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001059,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001357,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02761,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034226,"duration_s":0.046251,"completed_at":"2026-10-02T14:14:09.026Z","load_balancing_strategy":"primary","db_duration_s":0.008525,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1375gitlab # [ 292.985232] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:09.028Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"1bb095e68294d6c0f3fe9860","created_at":"2026-10-02T14:14:07.305Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-e0182bb6f1954479","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f377669ab5b36dd7ddbde6a7112f0876","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:07.325Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-1bb095e68294d6c0f3fe9860: start","job_status":"start","queue_duration_s":1.702645,"scheduling_latency_s":1.702663}1376gitlab # [ 293.006073] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:09.048Z","message":"No existing merge request to be cleaned up."}1377gitlab # [ 293.008999] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:09.059Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"1bb095e68294d6c0f3fe9860","created_at":"2026-10-02T14:14:07.305Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-e0182bb6f1954479","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f377669ab5b36dd7ddbde6a7112f0876","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:07.325Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-1bb095e68294d6c0f3fe9860: done: 0.031562 sec","job_status":"done","queue_duration_s":1.702645,"scheduling_latency_s":1.702663,"redis_calls":10,"redis_duration_s":0.002172,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000849,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001323,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019391,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031562,"completed_at":"2026-10-02T14:14:09.059Z","load_balancing_strategy":"primary","db_duration_s":0.010336,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1378gitlab # [ 293.066185] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:09.061Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"1c738a65f9b24cdd3b6e2973","created_at":"2026-10-02T14:14:07.305Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-e0182bb6f1954479","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f377669ab5b36dd7ddbde6a7112f0876","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:07.325Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-1c738a65f9b24cdd3b6e2973: start","job_status":"start","queue_duration_s":1.736126,"scheduling_latency_s":1.736144}1379gitlab # [ 293.080399] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:09.074Z","message":"No existing merge request to be cleaned up."}1380gitlab # [ 293.082276] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:09.085Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"1c738a65f9b24cdd3b6e2973","created_at":"2026-10-02T14:14:07.305Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-e0182bb6f1954479","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f377669ab5b36dd7ddbde6a7112f0876","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:07.325Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-1c738a65f9b24cdd3b6e2973: done: 0.023623 sec","job_status":"done","queue_duration_s":1.736126,"scheduling_latency_s":1.736144,"redis_calls":10,"redis_duration_s":0.002036,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00075,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001286,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016486,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023623,"completed_at":"2026-10-02T14:14:09.085Z","load_balancing_strategy":"primary","db_duration_s":0.00225,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1381gitlab # [ 293.123330] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:09.086Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e9b20fbb0711a3da3d03be20","created_at":"2026-10-02T14:14:07.305Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-e0182bb6f1954479","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f377669ab5b36dd7ddbde6a7112f0876","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:07.325Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e9b20fbb0711a3da3d03be20: start","job_status":"start","queue_duration_s":1.761302,"scheduling_latency_s":1.76132}1382gitlab # [ 293.136585] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:09.099Z","message":"No existing merge request to be cleaned up."}1383gitlab # [ 293.138402] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:14:09.109Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e9b20fbb0711a3da3d03be20","created_at":"2026-10-02T14:14:07.305Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-e0182bb6f1954479","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f377669ab5b36dd7ddbde6a7112f0876","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:07.325Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e9b20fbb0711a3da3d03be20: done: 0.022509 sec","job_status":"done","queue_duration_s":1.761302,"scheduling_latency_s":1.76132,"redis_calls":10,"redis_duration_s":0.001924,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000974,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00095,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016316,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022509,"completed_at":"2026-10-02T14:14:09.109Z","load_balancing_strategy":"primary","db_duration_s":0.001579,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1384gitlab # [ 309.191113] redis[788]: 10 changes in 300 seconds. Saving...1385gitlab # [ 309.196299] redis[788]: Background saving started by pid 35321386gitlab # [ 309.198919] redis[3532]: BGSAVE done, 425 keys saved, 0 keys skipped, 113405 bytes written.1387gitlab # [ 309.201923] redis[3532]: DB saved on disk1388gitlab # [ 309.203257] redis[3532]: Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB1389gitlab # [ 309.292083] redis[788]: Background saving terminated with success1390gitlab # [ 310.523748] postgres[857]: [857] LOG: checkpoint starting: time1391gitlab # [ 311.958155] gitaly[930]: time="2026-10-02T14:14:29.867Z" level=info msg="burden monitor stats" pid=9301392gitlab # [ 344.735479] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:02.643Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"8d2577d9fd15a31c50ddc13a","created_at":"2026-10-02T14:15:02.628Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"07480a557e5123768c3c44ddbd358767","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:02.640Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-8d2577d9fd15a31c50ddc13a: start","job_status":"start","queue_duration_s":0.003189,"scheduling_latency_s":0.003236}1393gitlab # [ 344.780620] setup-gitlab-start[3393]: W, [2026-10-02T14:15:02.689652 #3393] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1394gitlab # [ 344.816293] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:02.725Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1395gitlab # [ 344.910567] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:02.819Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"8d2577d9fd15a31c50ddc13a","created_at":"2026-10-02T14:15:02.628Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"07480a557e5123768c3c44ddbd358767","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:02.640Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-8d2577d9fd15a31c50ddc13a: done: 0.175967 sec","job_status":"done","queue_duration_s":0.003189,"scheduling_latency_s":0.003236,"redis_calls":13,"redis_duration_s":0.059154,"redis_read_bytes":773,"redis_write_bytes":1221,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.022407,"redis_feature_flag_read_bytes":768,"redis_feature_flag_write_bytes":400,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.008962,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.027785,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035084,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.175967,"completed_at":"2026-10-02T14:15:02.819Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1396gitlab # [ 344.978656] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:02.839Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"9b9885cc3b5798e68738ba69","created_at":"2026-10-02T14:15:02.666Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ae1036a38648b620bf5d8a76eb06db8d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:02.680Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-9b9885cc3b5798e68738ba69: start","job_status":"start","queue_duration_s":0.159026,"scheduling_latency_s":0.159073}1397gitlab # [ 345.045046] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:02.954Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"9b9885cc3b5798e68738ba69","created_at":"2026-10-02T14:15:02.666Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ae1036a38648b620bf5d8a76eb06db8d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:02.680Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-9b9885cc3b5798e68738ba69: done: 0.115158 sec","job_status":"done","queue_duration_s":0.159026,"scheduling_latency_s":0.159073,"redis_calls":8,"redis_duration_s":0.012014,"redis_read_bytes":133,"redis_write_bytes":846,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.001151,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":70,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.009369,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001494,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023575,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.115158,"completed_at":"2026-10-02T14:15:02.954Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1398gitlab # [ 345.113955] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:02.968Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"4c94326227ca9d72129fcf2f","created_at":"2026-10-02T14:15:02.715Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"212dc41b4917c019eafa565b8a62f227","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:02.728Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-4c94326227ca9d72129fcf2f: start","job_status":"start","queue_duration_s":0.23959,"scheduling_latency_s":0.239637}1399gitlab # [ 345.318147] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:03.227Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"4c94326227ca9d72129fcf2f","created_at":"2026-10-02T14:15:02.715Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"212dc41b4917c019eafa565b8a62f227","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:02.728Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-4c94326227ca9d72129fcf2f: done: 0.258743 sec","job_status":"done","queue_duration_s":0.23959,"scheduling_latency_s":0.239637,"redis_calls":9,"redis_duration_s":0.017539,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.009605,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.007934,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.095,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.095,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.058222,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0054454,"exclusive_lock_hold_duration_s":0.1737198,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.258743,"completed_at":"2026-10-02T14:15:03.227Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.141346,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1400gitlab # [ 345.386066] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:03.248Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"eeadfbb34707e7b733972b17","created_at":"2026-10-02T14:15:02.747Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"243487aed5605bea54fad7468b9da0e8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-02T14:15:02.764Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-eeadfbb34707e7b733972b17: start","job_status":"start","queue_duration_s":0.484718,"scheduling_latency_s":0.484768}1401gitlab # [ 345.417557] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:03.327Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1402gitlab # [ 345.438377] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:03.346Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"eeadfbb34707e7b733972b17","created_at":"2026-10-02T14:15:02.747Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"243487aed5605bea54fad7468b9da0e8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-02T14:15:02.764Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-eeadfbb34707e7b733972b17: done: 0.097258 sec","job_status":"done","queue_duration_s":0.484718,"scheduling_latency_s":0.484768,"redis_calls":7,"redis_duration_s":0.007985,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004264,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003721,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015115,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.097258,"completed_at":"2026-10-02T14:15:03.345Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1403gitlab # [ 345.496132] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:03.350Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"76a29409c81bdee5b664708b","created_at":"2026-10-02T14:15:02.809Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"befb5b93221ee72d07d61c6c0621024d","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:02.815Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-76a29409c81bdee5b664708b: start","job_status":"start","queue_duration_s":0.534779,"scheduling_latency_s":0.534814}1404gitlab # [ 345.511239] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:03.404Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"76a29409c81bdee5b664708b","created_at":"2026-10-02T14:15:02.809Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"befb5b93221ee72d07d61c6c0621024d","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:02.815Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-76a29409c81bdee5b664708b: done: 0.053917 sec","job_status":"done","queue_duration_s":0.534779,"scheduling_latency_s":0.534814,"redis_calls":8,"redis_duration_s":0.0053549999999999995,"redis_read_bytes":133,"redis_write_bytes":730,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000323,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":49,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00441,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000622,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021331,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.053917,"completed_at":"2026-10-02T14:15:03.404Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006134,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1405gitlab # [ 345.556754] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:03.407Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"130d06e01022ea393ff94e4d","created_at":"2026-10-02T14:15:02.837Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:02.845Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-130d06e01022ea393ff94e4d: start","job_status":"start","queue_duration_s":0.561878,"scheduling_latency_s":0.561913}1406gitlab # [ 345.693869] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:03.603Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"130d06e01022ea393ff94e4d","created_at":"2026-10-02T14:15:02.837Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:02.845Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-130d06e01022ea393ff94e4d: done: 0.195979 sec","job_status":"done","queue_duration_s":0.561878,"scheduling_latency_s":0.561913,"redis_calls":60,"redis_duration_s":0.022141,"redis_read_bytes":108,"redis_write_bytes":41109,"redis_queues_calls":2,"redis_queues_duration_s":0.000911,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.020443,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000787,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.140946,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.195979,"completed_at":"2026-10-02T14:15:03.603Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1407gitlab # [ 345.762894] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:03.605Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"7e3c1169f3e135a33580bf40","created_at":"2026-10-02T14:15:02.853Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"553092f66bddadb08732e98d7fa1be7e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-02T14:15:02.857Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-7e3c1169f3e135a33580bf40: start","job_status":"start","queue_duration_s":0.747523,"scheduling_latency_s":0.747554}1408gitlab # [ 345.780406] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:03.618Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1409gitlab # [ 345.784533] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:03.627Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"7e3c1169f3e135a33580bf40","created_at":"2026-10-02T14:15:02.853Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"553092f66bddadb08732e98d7fa1be7e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-02T14:15:02.857Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-7e3c1169f3e135a33580bf40: done: 0.022333 sec","job_status":"done","queue_duration_s":0.747523,"scheduling_latency_s":0.747554,"redis_calls":7,"redis_duration_s":0.002281,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001623,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000658,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013737,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022333,"completed_at":"2026-10-02T14:15:03.627Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1410gitlab # [ 345.825887] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:03.629Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"6674e0dd28f947b19b8b12e9","created_at":"2026-10-02T14:15:02.866Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cfbf8690bf8d00072c16487b5b2d4448","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:02.871Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-6674e0dd28f947b19b8b12e9: start","job_status":"start","queue_duration_s":0.758538,"scheduling_latency_s":0.758567}1411gitlab # [ 346.092320] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:04.001Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"6674e0dd28f947b19b8b12e9","created_at":"2026-10-02T14:15:02.866Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cfbf8690bf8d00072c16487b5b2d4448","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:02.871Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-6674e0dd28f947b19b8b12e9: done: 0.371836 sec","job_status":"done","queue_duration_s":0.758538,"scheduling_latency_s":0.758567,"redis_calls":9,"redis_duration_s":0.003015,"redis_read_bytes":7,"redis_write_bytes":987,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001633,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001382,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":129,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":129,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.044,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.044,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.318746,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00088798,"exclusive_lock_hold_duration_s":0.34702676,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.371836,"completed_at":"2026-10-02T14:15:04.001Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.069866,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1412gitlab # [ 346.159978] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:04.004Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"d0ad0111b79b8bf664a75ac8","created_at":"2026-10-02T14:15:02.881Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a2f009a012780e8d5dfe2fb8a29d5eb1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:02.885Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-d0ad0111b79b8bf664a75ac8: start","job_status":"start","queue_duration_s":1.118326,"scheduling_latency_s":1.118358}1413gitlab # [ 346.176794] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:04.033Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"d0ad0111b79b8bf664a75ac8","created_at":"2026-10-02T14:15:02.881Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a2f009a012780e8d5dfe2fb8a29d5eb1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:02.885Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-d0ad0111b79b8bf664a75ac8: done: 0.029651 sec","job_status":"done","queue_duration_s":1.118326,"scheduling_latency_s":1.118358,"redis_calls":9,"redis_duration_s":0.002796,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001389,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001407,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019005,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00089488,"exclusive_lock_hold_duration_s":0.00645824,"duration_s":0.029651,"completed_at":"2026-10-02T14:15:04.033Z","load_balancing_strategy":"primary","db_duration_s":0.002339,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1414gitlab # [ 346.220555] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:04.035Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"4453e731c11eb352fe10bfc5","created_at":"2026-10-02T14:15:02.893Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b5422880ff7a8faf172b52d3a541b4e8","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:02.919Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-4453e731c11eb352fe10bfc5: start","job_status":"start","queue_duration_s":1.116224,"scheduling_latency_s":1.116256}1415gitlab # [ 346.234965] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:04.085Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"4453e731c11eb352fe10bfc5","created_at":"2026-10-02T14:15:02.893Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b5422880ff7a8faf172b52d3a541b4e8","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:02.919Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-4453e731c11eb352fe10bfc5: done: 0.049958 sec","job_status":"done","queue_duration_s":1.116224,"scheduling_latency_s":1.116256,"redis_calls":10,"redis_duration_s":0.002998,"redis_read_bytes":9,"redis_write_bytes":1071,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000575,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001773,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00065,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030077,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00046784,"duration_s":0.049958,"completed_at":"2026-10-02T14:15:04.085Z","load_balancing_strategy":"primary","db_duration_s":0.010491,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1416gitlab # [ 346.279430] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:04.088Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"8e49600a92e857cb9ef5380e","created_at":"2026-10-02T14:15:02.939Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c55a78a2dfb1f19dd6d080616c9bad16","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:02.948Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-8e49600a92e857cb9ef5380e: start","job_status":"start","queue_duration_s":1.139185,"scheduling_latency_s":1.139217}1417gitlab # [ 346.296955] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:04.153Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"8e49600a92e857cb9ef5380e","created_at":"2026-10-02T14:15:02.939Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c55a78a2dfb1f19dd6d080616c9bad16","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:02.948Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-8e49600a92e857cb9ef5380e: done: 0.065677 sec","job_status":"done","queue_duration_s":1.139185,"scheduling_latency_s":1.139217,"redis_calls":9,"redis_duration_s":0.003516,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001931,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001585,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033424,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00137828,"exclusive_lock_hold_duration_s":0.0229824,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.065677,"completed_at":"2026-10-02T14:15:04.153Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.025044,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1418gitlab # [ 346.354124] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:04.155Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"f17ccd9991d0e356417f0bd6","created_at":"2026-10-02T14:15:02.966Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"92b6b582be5080fd63028f654aabd892","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:02.975Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-f17ccd9991d0e356417f0bd6: start","job_status":"start","queue_duration_s":1.180028,"scheduling_latency_s":1.18006}1419gitlab # [ 346.739374] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:04.648Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"f17ccd9991d0e356417f0bd6","created_at":"2026-10-02T14:15:02.966Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"92b6b582be5080fd63028f654aabd892","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:02.975Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-f17ccd9991d0e356417f0bd6: done: 0.493234 sec","job_status":"done","queue_duration_s":1.180028,"scheduling_latency_s":1.18006,"redis_calls":562,"redis_duration_s":0.119934,"redis_read_bytes":560,"redis_write_bytes":48760,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001275,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.118659,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.432796,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.493234,"completed_at":"2026-10-02T14:15:04.648Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1420gitlab # [ 346.804581] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:04.651Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"30d7b360a66eda994f2411d1","created_at":"2026-10-02T14:15:03.004Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"28e6d220a283ebb88c4510a4704fdd16","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.012Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-30d7b360a66eda994f2411d1: start","job_status":"start","queue_duration_s":1.63837,"scheduling_latency_s":1.638404}1421gitlab # [ 346.828821] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:04.673Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"30d7b360a66eda994f2411d1","created_at":"2026-10-02T14:15:03.004Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"28e6d220a283ebb88c4510a4704fdd16","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.012Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-30d7b360a66eda994f2411d1: done: 0.022123 sec","job_status":"done","queue_duration_s":1.63837,"scheduling_latency_s":1.638404,"redis_calls":7,"redis_duration_s":0.001818,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001287,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000531,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013937,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022123,"completed_at":"2026-10-02T14:15:04.673Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1422gitlab # [ 346.887902] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:04.675Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMigrateExternalDiffsWorker","jid":"135568f1f9ebd169de1cb495","created_at":"2026-10-02T14:15:03.024Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"815745b1e1ea9e48d6b2ea5691889a24","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1f56d9ee4ddd7fd390ae51dd41358b9c806acf2491cad4e28b1956ebf69a9b52","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.069Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMigrateExternalDiffsWorker JID-135568f1f9ebd169de1cb495: start","job_status":"start","queue_duration_s":1.605473,"scheduling_latency_s":1.605503}1423gitlab # [ 346.904601] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:04.707Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMigrateExternalDiffsWorker","jid":"135568f1f9ebd169de1cb495","created_at":"2026-10-02T14:15:03.024Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"815745b1e1ea9e48d6b2ea5691889a24","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1f56d9ee4ddd7fd390ae51dd41358b9c806acf2491cad4e28b1956ebf69a9b52","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.069Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMigrateExternalDiffsWorker JID-135568f1f9ebd169de1cb495: done: 0.032197 sec","job_status":"done","queue_duration_s":1.605473,"scheduling_latency_s":1.605503,"redis_calls":12,"redis_duration_s":0.003765,"redis_read_bytes":11,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00059,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":311,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001905,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00127,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":533,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02024,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00092202,"exclusive_lock_hold_duration_s":0.0007837200000000001,"duration_s":0.032197,"completed_at":"2026-10-02T14:15:04.707Z","load_balancing_strategy":"primary","db_duration_s":0.000777,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1424gitlab # [ 346.955531] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:04.709Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"40a06800b2af160a2ea924c1","created_at":"2026-10-02T14:15:03.079Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cf6f0469c441e82ebfa465ccc10960fa","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.095Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-40a06800b2af160a2ea924c1: start","job_status":"start","queue_duration_s":1.613588,"scheduling_latency_s":1.613619}1425gitlab # [ 346.970892] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:04.749Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"40a06800b2af160a2ea924c1","created_at":"2026-10-02T14:15:03.079Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cf6f0469c441e82ebfa465ccc10960fa","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.095Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-40a06800b2af160a2ea924c1: done: 0.039983 sec","job_status":"done","queue_duration_s":1.613588,"scheduling_latency_s":1.613619,"redis_calls":11,"redis_duration_s":0.003793,"redis_read_bytes":9,"redis_write_bytes":1174,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000983,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001544,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000629,"redis_rate_limiting_write_bytes":51,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000637,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027043,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00046736,"extra.cells_stale_requests_cleanup_cron_worker.removed_count":0,"duration_s":0.039983,"completed_at":"2026-10-02T14:15:04.749Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00092,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1426gitlab # [ 347.017189] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:04.752Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"e947796bf94e5898744ea3c6","created_at":"2026-10-02T14:15:03.101Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7e95313629e24704fd3141afc93e751a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.107Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-e947796bf94e5898744ea3c6: start","job_status":"start","queue_duration_s":1.645285,"scheduling_latency_s":1.645325}1427gitlab # [ 347.032473] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:04.768Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1428gitlab # [ 347.035777] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:04.778Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"e947796bf94e5898744ea3c6","created_at":"2026-10-02T14:15:03.101Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7e95313629e24704fd3141afc93e751a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.107Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-e947796bf94e5898744ea3c6: done: 0.026172 sec","job_status":"done","queue_duration_s":1.645285,"scheduling_latency_s":1.645325,"redis_calls":7,"redis_duration_s":0.002064,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001484,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00058,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0163,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026172,"completed_at":"2026-10-02T14:15:04.778Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1429gitlab # [ 347.078491] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:04.781Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"54e67510778e41c003109c5a","created_at":"2026-10-02T14:15:03.112Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8b5137f38820b9f1992a84ceafed3af0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.122Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-54e67510778e41c003109c5a: start","job_status":"start","queue_duration_s":1.658204,"scheduling_latency_s":1.658238}1430gitlab # [ 347.093638] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:04.795Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1431gitlab # [ 347.097620] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:04.805Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"54e67510778e41c003109c5a","created_at":"2026-10-02T14:15:03.112Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8b5137f38820b9f1992a84ceafed3af0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.122Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-54e67510778e41c003109c5a: done: 0.024398 sec","job_status":"done","queue_duration_s":1.658204,"scheduling_latency_s":1.658238,"redis_calls":7,"redis_duration_s":0.002048,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001486,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000562,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014893,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024398,"completed_at":"2026-10-02T14:15:04.805Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1432gitlab # [ 347.141380] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:04.807Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"97332490e6b1953dc2029a02","created_at":"2026-10-02T14:15:03.172Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5b11b76de46096138a529192939d9c18","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.187Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-97332490e6b1953dc2029a02: start","job_status":"start","queue_duration_s":1.620591,"scheduling_latency_s":1.620624}1433gitlab # [ 347.156133] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:04.836Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"97332490e6b1953dc2029a02","created_at":"2026-10-02T14:15:03.172Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5b11b76de46096138a529192939d9c18","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.187Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-97332490e6b1953dc2029a02: done: 0.029351 sec","job_status":"done","queue_duration_s":1.620591,"scheduling_latency_s":1.620624,"redis_calls":9,"redis_duration_s":0.0028840000000000003,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001526,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001358,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018871,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00088026,"exclusive_lock_hold_duration_s":0.0052712,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.029351,"completed_at":"2026-10-02T14:15:04.836Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001798,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1434gitlab # [ 347.199803] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:04.839Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckMergeJobsWorker","jid":"b3724239ed9aae2e1b41814b","created_at":"2026-10-02T14:15:03.213Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"59987c143330dcd58821458fb94a8a0a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:ce304458c2d63d5d84855b65f51692704db52bf763a718b10c08a2bd5f588f06","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.233Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckMergeJobsWorker JID-b3724239ed9aae2e1b41814b: start","job_status":"start","queue_duration_s":1.605648,"scheduling_latency_s":1.60568}1435gitlab # [ 347.213650] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:04.884Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckMergeJobsWorker","jid":"b3724239ed9aae2e1b41814b","created_at":"2026-10-02T14:15:03.213Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"59987c143330dcd58821458fb94a8a0a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:ce304458c2d63d5d84855b65f51692704db52bf763a718b10c08a2bd5f588f06","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.233Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckMergeJobsWorker JID-b3724239ed9aae2e1b41814b: done: 0.045381 sec","job_status":"done","queue_duration_s":1.605648,"scheduling_latency_s":1.60568,"redis_calls":11,"redis_duration_s":0.003502,"redis_read_bytes":10,"redis_write_bytes":1022,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000656,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":283,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001772,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001074,"redis_shared_state_read_bytes":4,"redis_shared_state_write_bytes":225,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028687,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00044668,"duration_s":0.045381,"completed_at":"2026-10-02T14:15:04.884Z","load_balancing_strategy":"primary","db_duration_s":0.006255,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1436gitlab # [ 347.255911] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:04.886Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainVerificationCronWorker","jid":"d4fe9ef9e6cb56e87b114eac","created_at":"2026-10-02T14:15:03.240Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d9c00c86b2009a448eb7ad9859d21a7e","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:4b3095972298b58b9c376f502255d63e0934f66e5921dfe9547414a5f347be92","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.256Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainVerificationCronWorker JID-d4fe9ef9e6cb56e87b114eac: start","job_status":"start","queue_duration_s":1.630493,"scheduling_latency_s":1.630529}1437gitlab # [ 347.269910] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:04.938Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainVerificationCronWorker","jid":"d4fe9ef9e6cb56e87b114eac","created_at":"2026-10-02T14:15:03.240Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d9c00c86b2009a448eb7ad9859d21a7e","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:4b3095972298b58b9c376f502255d63e0934f66e5921dfe9547414a5f347be92","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.256Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainVerificationCronWorker JID-d4fe9ef9e6cb56e87b114eac: done: 0.051575 sec","job_status":"done","queue_duration_s":1.630493,"scheduling_latency_s":1.630529,"redis_calls":10,"redis_duration_s":0.0031060000000000003,"redis_read_bytes":9,"redis_write_bytes":1097,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000638,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":309,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001722,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000746,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":218,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033154,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00049898,"duration_s":0.051575,"completed_at":"2026-10-02T14:15:04.938Z","load_balancing_strategy":"primary","db_duration_s":0.006331,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1438gitlab # [ 347.313953] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:04.940Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"66c7fa5a5b9b071ea8c1c02c","created_at":"2026-10-02T14:15:03.284Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f56f7ec9afe1a2b511f35c2a1685a7a0","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.288Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-66c7fa5a5b9b071ea8c1c02c: start","job_status":"start","queue_duration_s":1.651769,"scheduling_latency_s":1.651807}1439gitlab # [ 347.328328] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:04.979Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"66c7fa5a5b9b071ea8c1c02c","created_at":"2026-10-02T14:15:03.284Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f56f7ec9afe1a2b511f35c2a1685a7a0","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.288Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-66c7fa5a5b9b071ea8c1c02c: done: 0.039086 sec","job_status":"done","queue_duration_s":1.651769,"scheduling_latency_s":1.651807,"redis_calls":7,"redis_duration_s":0.002373,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001726,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000647,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018326,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.039086,"completed_at":"2026-10-02T14:15:04.979Z","load_balancing_strategy":"primary","db_duration_s":0.011925,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1440gitlab # [ 347.371007] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:04.981Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Gitlab::Import::StuckProjectImportJobsWorker","jid":"d01f399344a04d594784eca1","created_at":"2026-10-02T14:15:03.291Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"232645efe5387ab841edbccd72257971","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:65d1a1bda9b2a0115065de4f20868193e50a174305a7166fa20fbc82759e43e4","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.299Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Gitlab::Import::StuckProjectImportJobsWorker JID-d01f399344a04d594784eca1: start","job_status":"start","queue_duration_s":1.681987,"scheduling_latency_s":1.682023}1441gitlab # [ 347.384962] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.020Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Gitlab::Import::StuckProjectImportJobsWorker","jid":"d01f399344a04d594784eca1","created_at":"2026-10-02T14:15:03.291Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"232645efe5387ab841edbccd72257971","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:65d1a1bda9b2a0115065de4f20868193e50a174305a7166fa20fbc82759e43e4","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.299Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Gitlab::Import::StuckProjectImportJobsWorker JID-d01f399344a04d594784eca1: done: 0.03875 sec","job_status":"done","queue_duration_s":1.681987,"scheduling_latency_s":1.682023,"redis_calls":10,"redis_duration_s":0.002484,"redis_read_bytes":9,"redis_write_bytes":1173,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000569,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001356,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000559,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":236,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023766,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00031502,"duration_s":0.03875,"completed_at":"2026-10-02T14:15:05.020Z","load_balancing_strategy":"primary","db_duration_s":0.004621,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1442gitlab # [ 347.427730] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.021Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Issues::RescheduleStuckIssueRebalancesWorker","jid":"e7e2805eb879680812d14700","created_at":"2026-10-02T14:15:03.308Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0280ee6e21546447f8fc8d2ee643dd0c","meta.root_caller_id":"Cronjob","meta.feature_category":"team_planning","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fc2fa70b35c572764e1a8fbcbc4fdd3df48ec5a5855870d65915560d23915134","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.320Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Issues::RescheduleStuckIssueRebalancesWorker JID-e7e2805eb879680812d14700: start","job_status":"start","queue_duration_s":1.701232,"scheduling_latency_s":1.701258}1443gitlab # [ 347.442955] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.042Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Issues::RescheduleStuckIssueRebalancesWorker","jid":"e7e2805eb879680812d14700","created_at":"2026-10-02T14:15:03.308Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0280ee6e21546447f8fc8d2ee643dd0c","meta.root_caller_id":"Cronjob","meta.feature_category":"team_planning","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fc2fa70b35c572764e1a8fbcbc4fdd3df48ec5a5855870d65915560d23915134","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.320Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Issues::RescheduleStuckIssueRebalancesWorker JID-e7e2805eb879680812d14700: done: 0.020658 sec","job_status":"done","queue_duration_s":1.701232,"scheduling_latency_s":1.701258,"redis_calls":11,"redis_duration_s":0.001783,"redis_read_bytes":9,"redis_write_bytes":1239,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000338,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00093,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":610,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000515,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":298,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012584,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00030738,"duration_s":0.020658,"completed_at":"2026-10-02T14:15:05.042Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000539,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1444gitlab # [ 347.486919] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.044Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"e761661136fdc55f0be341e4","created_at":"2026-10-02T14:15:03.335Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"33b702ed99e0de2b3c4e8ab2f979d533","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.341Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-e761661136fdc55f0be341e4: start","job_status":"start","queue_duration_s":1.702251,"scheduling_latency_s":1.702279}1445gitlab # [ 347.501094] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.102Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"e761661136fdc55f0be341e4","created_at":"2026-10-02T14:15:03.335Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"33b702ed99e0de2b3c4e8ab2f979d533","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.341Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-e761661136fdc55f0be341e4: done: 0.058641 sec","job_status":"done","queue_duration_s":1.702251,"scheduling_latency_s":1.702279,"redis_calls":14,"redis_duration_s":0.002454,"redis_read_bytes":16,"redis_write_bytes":4030,"redis_queues_calls":2,"redis_queues_duration_s":0.000254,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001695,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000505,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039943,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.058641,"completed_at":"2026-10-02T14:15:05.102Z","load_balancing_strategy":"primary","db_duration_s":0.012703,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1446gitlab # [ 347.544935] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.104Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"01f79dccc113fb67b612badf","created_at":"2026-10-02T14:15:03.361Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cb9b4b81808c280f64c407f5cf478b2e","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.372Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-01f79dccc113fb67b612badf: start","job_status":"start","queue_duration_s":1.731149,"scheduling_latency_s":1.731173}1447gitlab # [ 347.559590] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.127Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"01f79dccc113fb67b612badf","created_at":"2026-10-02T14:15:03.361Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cb9b4b81808c280f64c407f5cf478b2e","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.372Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-01f79dccc113fb67b612badf: done: 0.023128 sec","job_status":"done","queue_duration_s":1.731149,"scheduling_latency_s":1.731173,"redis_calls":7,"redis_duration_s":0.001273,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000925,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000348,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013396,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023128,"completed_at":"2026-10-02T14:15:05.127Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004277,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1448gitlab # [ 347.601598] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.128Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"420865a9e2db215846afe90d","created_at":"2026-10-02T14:15:03.396Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3b74f6f56d7378b30580bb1a5c4a0a31","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-02T14:15:03.401Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-420865a9e2db215846afe90d: start","job_status":"start","queue_duration_s":1.727362,"scheduling_latency_s":1.727388}1449gitlab # [ 347.615388] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.151Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"420865a9e2db215846afe90d","created_at":"2026-10-02T14:15:03.396Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3b74f6f56d7378b30580bb1a5c4a0a31","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-02T14:15:03.401Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-420865a9e2db215846afe90d: done: 0.023065 sec","job_status":"done","queue_duration_s":1.727362,"scheduling_latency_s":1.727388,"redis_calls":7,"redis_duration_s":0.001246,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000923,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000323,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015749,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023065,"completed_at":"2026-10-02T14:15:05.151Z","load_balancing_strategy":"primary","db_duration_s":0.001722,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1450gitlab # [ 347.656342] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.153Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"4cf516ab39ea94c5eba7e25c","created_at":"2026-10-02T14:15:03.418Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d38442956c03664e0967682373d1bfed","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.422Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-4cf516ab39ea94c5eba7e25c: start","job_status":"start","queue_duration_s":1.730413,"scheduling_latency_s":1.730438}1451gitlab # [ 347.671524] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.161Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1452gitlab # [ 347.675245] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.168Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"4cf516ab39ea94c5eba7e25c","created_at":"2026-10-02T14:15:03.418Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d38442956c03664e0967682373d1bfed","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.422Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-4cf516ab39ea94c5eba7e25c: done: 0.015009 sec","job_status":"done","queue_duration_s":1.730413,"scheduling_latency_s":1.730438,"redis_calls":7,"redis_duration_s":0.00117,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00085,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00032,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009388,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015009,"completed_at":"2026-10-02T14:15:05.168Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1453gitlab # [ 347.718811] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.169Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"3a42cc63e7ec81f903fbbc7b","created_at":"2026-10-02T14:15:03.456Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"80b409bdbe043d206540b03bd8f8fec2","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.466Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-3a42cc63e7ec81f903fbbc7b: start","job_status":"start","queue_duration_s":1.702537,"scheduling_latency_s":1.70256}1454gitlab # [ 347.734051] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.190Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"3a42cc63e7ec81f903fbbc7b","created_at":"2026-10-02T14:15:03.456Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-acb269d7d1454ac7","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"80b409bdbe043d206540b03bd8f8fec2","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.466Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-3a42cc63e7ec81f903fbbc7b: done: 0.021435 sec","job_status":"done","queue_duration_s":1.702537,"scheduling_latency_s":1.70256,"redis_calls":10,"redis_duration_s":0.0016519999999999998,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00094,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000712,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014237,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00541974,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.021435,"completed_at":"2026-10-02T14:15:05.190Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002552,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1455gitlab # [ 347.778556] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.192Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2ce7871aa2ab1691034d0a1b","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2ce7871aa2ab1691034d0a1b: start","job_status":"start","queue_duration_s":1.600207,"scheduling_latency_s":1.600224}1456gitlab # [ 347.791405] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.207Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2ce7871aa2ab1691034d0a1b","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2ce7871aa2ab1691034d0a1b: done: 0.015688 sec","job_status":"done","queue_duration_s":1.600207,"scheduling_latency_s":1.600224,"redis_calls":13,"redis_duration_s":0.001694,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000593,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001101,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010551,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015688,"completed_at":"2026-10-02T14:15:05.207Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1457gitlab # [ 347.833455] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.209Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"620e5e1fdbbb4757736e39c5","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-620e5e1fdbbb4757736e39c5: start","job_status":"start","queue_duration_s":1.617366,"scheduling_latency_s":1.617383}1458gitlab # [ 347.853863] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.225Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"620e5e1fdbbb4757736e39c5","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-620e5e1fdbbb4757736e39c5: done: 0.015898 sec","job_status":"done","queue_duration_s":1.617366,"scheduling_latency_s":1.617383,"redis_calls":13,"redis_duration_s":0.0017300000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000671,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001059,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010673,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015898,"completed_at":"2026-10-02T14:15:05.225Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1459gitlab # [ 347.913975] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.226Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7ecf61697e87d81b28929ee1","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7ecf61697e87d81b28929ee1: start","job_status":"start","queue_duration_s":1.634445,"scheduling_latency_s":1.634462}1460gitlab # [ 347.928662] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.241Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7ecf61697e87d81b28929ee1","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7ecf61697e87d81b28929ee1: done: 0.015228 sec","job_status":"done","queue_duration_s":1.634445,"scheduling_latency_s":1.634462,"redis_calls":13,"redis_duration_s":0.001635,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00058,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001055,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010361,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015228,"completed_at":"2026-10-02T14:15:05.241Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1461gitlab # [ 347.972319] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.242Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad1ad54805022cf5e5b28f18","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad1ad54805022cf5e5b28f18: start","job_status":"start","queue_duration_s":1.650925,"scheduling_latency_s":1.650942}1462gitlab # [ 347.986087] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.257Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad1ad54805022cf5e5b28f18","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad1ad54805022cf5e5b28f18: done: 0.015058 sec","job_status":"done","queue_duration_s":1.650925,"scheduling_latency_s":1.650942,"redis_calls":13,"redis_duration_s":0.001581,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000554,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001027,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010293,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015058,"completed_at":"2026-10-02T14:15:05.257Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1463gitlab # [ 348.027601] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.259Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"541182359de468f1909e5ab0","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-541182359de468f1909e5ab0: start","job_status":"start","queue_duration_s":1.667169,"scheduling_latency_s":1.667186}1464gitlab # [ 348.040678] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.273Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"541182359de468f1909e5ab0","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-541182359de468f1909e5ab0: done: 0.01474 sec","job_status":"done","queue_duration_s":1.667169,"scheduling_latency_s":1.667186,"redis_calls":13,"redis_duration_s":0.001592,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000562,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00103,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010176,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01474,"completed_at":"2026-10-02T14:15:05.273Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1465gitlab # [ 348.081119] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.275Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"008c7be6d297f31191defe0d","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-008c7be6d297f31191defe0d: start","job_status":"start","queue_duration_s":1.683095,"scheduling_latency_s":1.683113}1466gitlab # [ 348.093802] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.289Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"008c7be6d297f31191defe0d","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-008c7be6d297f31191defe0d: done: 0.014698 sec","job_status":"done","queue_duration_s":1.683095,"scheduling_latency_s":1.683113,"redis_calls":13,"redis_duration_s":0.001578,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000567,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001011,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010163,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014698,"completed_at":"2026-10-02T14:15:05.289Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1467gitlab # [ 348.135616] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.290Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1e6b87cc6e7b286a62c3247c","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1e6b87cc6e7b286a62c3247c: start","job_status":"start","queue_duration_s":1.698995,"scheduling_latency_s":1.699011}1468gitlab # [ 348.148747] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.305Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1e6b87cc6e7b286a62c3247c","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1e6b87cc6e7b286a62c3247c: done: 0.014752 sec","job_status":"done","queue_duration_s":1.698995,"scheduling_latency_s":1.699011,"redis_calls":13,"redis_duration_s":0.001607,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000575,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001032,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010236,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014752,"completed_at":"2026-10-02T14:15:05.305Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1469gitlab # [ 348.189087] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.306Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"60ec4ea3e007791d0041fab4","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-60ec4ea3e007791d0041fab4: start","job_status":"start","queue_duration_s":1.714972,"scheduling_latency_s":1.71499}1470gitlab # [ 348.201997] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.371Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"60ec4ea3e007791d0041fab4","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-60ec4ea3e007791d0041fab4: done: 0.06434 sec","job_status":"done","queue_duration_s":1.714972,"scheduling_latency_s":1.71499,"redis_calls":13,"redis_duration_s":0.0017859999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000635,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001151,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.058443,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.06434,"completed_at":"2026-10-02T14:15:05.371Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1471gitlab # [ 348.242547] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.372Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"327a05a31204920e67d3d6cd","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-327a05a31204920e67d3d6cd: start","job_status":"start","queue_duration_s":1.780531,"scheduling_latency_s":1.78055}1472gitlab # [ 348.255408] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.387Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"327a05a31204920e67d3d6cd","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-327a05a31204920e67d3d6cd: done: 0.015104 sec","job_status":"done","queue_duration_s":1.780531,"scheduling_latency_s":1.78055,"redis_calls":13,"redis_duration_s":0.0016879999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000615,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001073,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010252,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015104,"completed_at":"2026-10-02T14:15:05.387Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1473gitlab # [ 348.296054] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.388Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f8b697908a7a86a60a7f73d6","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f8b697908a7a86a60a7f73d6: start","job_status":"start","queue_duration_s":1.796853,"scheduling_latency_s":1.796871}1474gitlab # [ 348.308972] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.405Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f8b697908a7a86a60a7f73d6","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f8b697908a7a86a60a7f73d6: done: 0.016347 sec","job_status":"done","queue_duration_s":1.796853,"scheduling_latency_s":1.796871,"redis_calls":13,"redis_duration_s":0.001657,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000598,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001059,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01004,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016347,"completed_at":"2026-10-02T14:15:05.405Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1475gitlab # [ 348.349155] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.406Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c81779c98a380845287b435f","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c81779c98a380845287b435f: start","job_status":"start","queue_duration_s":1.814385,"scheduling_latency_s":1.814402}1476gitlab # [ 348.363152] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.422Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c81779c98a380845287b435f","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c81779c98a380845287b435f: done: 0.016166 sec","job_status":"done","queue_duration_s":1.814385,"scheduling_latency_s":1.814402,"redis_calls":13,"redis_duration_s":0.00165,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000608,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001042,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011294,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016166,"completed_at":"2026-10-02T14:15:05.422Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1477gitlab # [ 348.403527] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.423Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"facc3a988023e90e8dd516fa","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-facc3a988023e90e8dd516fa: start","job_status":"start","queue_duration_s":1.831675,"scheduling_latency_s":1.831693}1478gitlab # [ 348.416617] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.439Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"facc3a988023e90e8dd516fa","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-facc3a988023e90e8dd516fa: done: 0.015717 sec","job_status":"done","queue_duration_s":1.831675,"scheduling_latency_s":1.831693,"redis_calls":13,"redis_duration_s":0.0016099999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000573,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001037,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011109,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015717,"completed_at":"2026-10-02T14:15:05.439Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1479gitlab # [ 348.457150] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.440Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"507b804a1f493b0a27094ed2","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-507b804a1f493b0a27094ed2: start","job_status":"start","queue_duration_s":1.848521,"scheduling_latency_s":1.848538}1480gitlab # [ 348.469962] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.455Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"507b804a1f493b0a27094ed2","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-507b804a1f493b0a27094ed2: done: 0.015499 sec","job_status":"done","queue_duration_s":1.848521,"scheduling_latency_s":1.848538,"redis_calls":13,"redis_duration_s":0.001572,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000567,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001005,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01092,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015499,"completed_at":"2026-10-02T14:15:05.455Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1481gitlab # [ 348.510180] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.457Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4626d14cf381ceaaf932faf6","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4626d14cf381ceaaf932faf6: start","job_status":"start","queue_duration_s":1.865226,"scheduling_latency_s":1.865244}1482gitlab # [ 348.523097] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.475Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4626d14cf381ceaaf932faf6","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4626d14cf381ceaaf932faf6: done: 0.01842 sec","job_status":"done","queue_duration_s":1.865226,"scheduling_latency_s":1.865244,"redis_calls":13,"redis_duration_s":0.001608,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000592,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001016,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010175,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01842,"completed_at":"2026-10-02T14:15:05.475Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1483gitlab # [ 348.564483] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.476Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"757dc65c6aa662fd620a8708","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-757dc65c6aa662fd620a8708: start","job_status":"start","queue_duration_s":1.884805,"scheduling_latency_s":1.884821}1484gitlab # [ 348.577241] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.492Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"757dc65c6aa662fd620a8708","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-757dc65c6aa662fd620a8708: done: 0.015738 sec","job_status":"done","queue_duration_s":1.884805,"scheduling_latency_s":1.884821,"redis_calls":13,"redis_duration_s":0.001692,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000575,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001117,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011132,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015738,"completed_at":"2026-10-02T14:15:05.492Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1485gitlab # [ 348.617464] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.493Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ad73c65953f78490ab3323e","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ad73c65953f78490ab3323e: start","job_status":"start","queue_duration_s":1.901656,"scheduling_latency_s":1.901673}1486gitlab # [ 348.630608] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.509Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ad73c65953f78490ab3323e","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ad73c65953f78490ab3323e: done: 0.015559 sec","job_status":"done","queue_duration_s":1.901656,"scheduling_latency_s":1.901673,"redis_calls":13,"redis_duration_s":0.001669,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000578,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001091,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010962,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015559,"completed_at":"2026-10-02T14:15:05.509Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1487gitlab # [ 348.671493] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.510Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d7676d18d7bb96362ad09b36","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d7676d18d7bb96362ad09b36: start","job_status":"start","queue_duration_s":1.91867,"scheduling_latency_s":1.918688}1488gitlab # [ 348.684322] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.527Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d7676d18d7bb96362ad09b36","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d7676d18d7bb96362ad09b36: done: 0.017091 sec","job_status":"done","queue_duration_s":1.91867,"scheduling_latency_s":1.918688,"redis_calls":13,"redis_duration_s":0.001564,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000562,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001002,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01011,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017091,"completed_at":"2026-10-02T14:15:05.527Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1489gitlab # [ 348.725239] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.528Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dcd6d185d2def4614fc41770","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dcd6d185d2def4614fc41770: start","job_status":"start","queue_duration_s":1.936891,"scheduling_latency_s":1.936907}1490gitlab # [ 348.738127] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.544Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dcd6d185d2def4614fc41770","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dcd6d185d2def4614fc41770: done: 0.015555 sec","job_status":"done","queue_duration_s":1.936891,"scheduling_latency_s":1.936907,"redis_calls":13,"redis_duration_s":0.0016120000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000574,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001038,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010989,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015555,"completed_at":"2026-10-02T14:15:05.544Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1491gitlab # [ 348.779863] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.545Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c1e2d2e79dd987abd7447df5","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c1e2d2e79dd987abd7447df5: start","job_status":"start","queue_duration_s":1.953587,"scheduling_latency_s":1.953604}1492gitlab # [ 348.792903] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.562Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c1e2d2e79dd987abd7447df5","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c1e2d2e79dd987abd7447df5: done: 0.016802 sec","job_status":"done","queue_duration_s":1.953587,"scheduling_latency_s":1.953604,"redis_calls":13,"redis_duration_s":0.001654,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000642,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001012,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012193,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016802,"completed_at":"2026-10-02T14:15:05.562Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1493gitlab # [ 348.833904] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.563Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b8ab91f95dceae1dd368828f","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b8ab91f95dceae1dd368828f: start","job_status":"start","queue_duration_s":1.97158,"scheduling_latency_s":1.971597}1494gitlab # [ 348.846864] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.579Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b8ab91f95dceae1dd368828f","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b8ab91f95dceae1dd368828f: done: 0.015653 sec","job_status":"done","queue_duration_s":1.97158,"scheduling_latency_s":1.971597,"redis_calls":13,"redis_duration_s":0.0015860000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000599,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000987,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010037,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015653,"completed_at":"2026-10-02T14:15:05.579Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1495gitlab # [ 348.887667] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.581Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8f14c571e4d478ffc01b6f01","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8f14c571e4d478ffc01b6f01: start","job_status":"start","queue_duration_s":1.989194,"scheduling_latency_s":1.989212}1496gitlab # [ 348.900930] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.597Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8f14c571e4d478ffc01b6f01","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8f14c571e4d478ffc01b6f01: done: 0.016625 sec","job_status":"done","queue_duration_s":1.989194,"scheduling_latency_s":1.989212,"redis_calls":13,"redis_duration_s":0.0017029999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000601,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001102,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01104,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016625,"completed_at":"2026-10-02T14:15:05.597Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1497gitlab # [ 348.941395] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.598Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"26766e2cda4136c74614c94b","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-26766e2cda4136c74614c94b: start","job_status":"start","queue_duration_s":2.007006,"scheduling_latency_s":2.007023}1498gitlab # [ 348.954474] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.614Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"26766e2cda4136c74614c94b","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-26766e2cda4136c74614c94b: done: 0.015999 sec","job_status":"done","queue_duration_s":2.007006,"scheduling_latency_s":2.007023,"redis_calls":13,"redis_duration_s":0.002418,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000581,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001837,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01121,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015999,"completed_at":"2026-10-02T14:15:05.614Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1499gitlab # [ 348.996213] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.616Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb40e87be45d71710091afa0","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb40e87be45d71710091afa0: start","job_status":"start","queue_duration_s":2.024228,"scheduling_latency_s":2.024245}1500gitlab # [ 349.009170] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.631Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb40e87be45d71710091afa0","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb40e87be45d71710091afa0: done: 0.015736 sec","job_status":"done","queue_duration_s":2.024228,"scheduling_latency_s":2.024245,"redis_calls":13,"redis_duration_s":0.0016279999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000606,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001022,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011096,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015736,"completed_at":"2026-10-02T14:15:05.631Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1501gitlab # [ 349.050092] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.633Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c41efeae656e0bf5a0b80a62","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c41efeae656e0bf5a0b80a62: start","job_status":"start","queue_duration_s":2.041139,"scheduling_latency_s":2.041156}1502gitlab # [ 349.063087] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.648Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c41efeae656e0bf5a0b80a62","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c41efeae656e0bf5a0b80a62: done: 0.015731 sec","job_status":"done","queue_duration_s":2.041139,"scheduling_latency_s":2.041156,"redis_calls":13,"redis_duration_s":0.001755,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000668,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001087,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010215,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015731,"completed_at":"2026-10-02T14:15:05.648Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1503gitlab # [ 349.103970] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.650Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3909953d16160edbf80045b7","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3909953d16160edbf80045b7: start","job_status":"start","queue_duration_s":2.05901,"scheduling_latency_s":2.059027}1504gitlab # [ 349.117125] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.666Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3909953d16160edbf80045b7","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3909953d16160edbf80045b7: done: 0.016009 sec","job_status":"done","queue_duration_s":2.05901,"scheduling_latency_s":2.059027,"redis_calls":13,"redis_duration_s":0.001603,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000595,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001008,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011203,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016009,"completed_at":"2026-10-02T14:15:05.666Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1505gitlab # [ 349.157747] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.668Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"23a21563cf0cddfef738f376","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-23a21563cf0cddfef738f376: start","job_status":"start","queue_duration_s":2.076126,"scheduling_latency_s":2.076142}1506gitlab # [ 349.171639] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.683Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"23a21563cf0cddfef738f376","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-23a21563cf0cddfef738f376: done: 0.01584 sec","job_status":"done","queue_duration_s":2.076126,"scheduling_latency_s":2.076142,"redis_calls":13,"redis_duration_s":0.00162,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000568,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001052,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0112,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01584,"completed_at":"2026-10-02T14:15:05.683Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1507gitlab # [ 349.212523] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.685Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e8b7007c33591e0abc5d788","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e8b7007c33591e0abc5d788: start","job_status":"start","queue_duration_s":2.09315,"scheduling_latency_s":2.093166}1508gitlab # [ 349.225196] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.700Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e8b7007c33591e0abc5d788","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e8b7007c33591e0abc5d788: done: 0.015533 sec","job_status":"done","queue_duration_s":2.09315,"scheduling_latency_s":2.093166,"redis_calls":13,"redis_duration_s":0.001614,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000562,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001052,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010986,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015533,"completed_at":"2026-10-02T14:15:05.700Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1509gitlab # [ 349.265727] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.701Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6dae58a45bf86ddbce2eee3b","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6dae58a45bf86ddbce2eee3b: start","job_status":"start","queue_duration_s":2.109801,"scheduling_latency_s":2.109818}1510gitlab # [ 349.278624] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.717Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6dae58a45bf86ddbce2eee3b","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6dae58a45bf86ddbce2eee3b: done: 0.015969 sec","job_status":"done","queue_duration_s":2.109801,"scheduling_latency_s":2.109818,"redis_calls":13,"redis_duration_s":0.001787,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000549,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001238,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010132,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015969,"completed_at":"2026-10-02T14:15:05.717Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1511gitlab # [ 349.319664] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.720Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84989b2cd1c3bef6f03e42f4","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84989b2cd1c3bef6f03e42f4: start","job_status":"start","queue_duration_s":2.128056,"scheduling_latency_s":2.128072}1512gitlab # [ 349.332731] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.735Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84989b2cd1c3bef6f03e42f4","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84989b2cd1c3bef6f03e42f4: done: 0.015839 sec","job_status":"done","queue_duration_s":2.128056,"scheduling_latency_s":2.128072,"redis_calls":13,"redis_duration_s":0.001611,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000594,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001017,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01122,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015839,"completed_at":"2026-10-02T14:15:05.735Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1513gitlab # [ 349.374067] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.736Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0822ef8f3b12f56f092f4fa0","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0822ef8f3b12f56f092f4fa0: start","job_status":"start","queue_duration_s":2.145001,"scheduling_latency_s":2.145017}1514gitlab # [ 349.386927] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.752Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0822ef8f3b12f56f092f4fa0","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0822ef8f3b12f56f092f4fa0: done: 0.016035 sec","job_status":"done","queue_duration_s":2.145001,"scheduling_latency_s":2.145017,"redis_calls":13,"redis_duration_s":0.0017209999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00064,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001081,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011407,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016035,"completed_at":"2026-10-02T14:15:05.752Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1515gitlab # [ 349.427908] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.754Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eddd7f99f4929c57652894f9","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eddd7f99f4929c57652894f9: start","job_status":"start","queue_duration_s":2.162163,"scheduling_latency_s":2.16218}1516gitlab # [ 349.440784] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.770Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eddd7f99f4929c57652894f9","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eddd7f99f4929c57652894f9: done: 0.016116 sec","job_status":"done","queue_duration_s":2.162163,"scheduling_latency_s":2.16218,"redis_calls":13,"redis_duration_s":0.001637,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000588,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001049,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011528,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016116,"completed_at":"2026-10-02T14:15:05.770Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1517gitlab # [ 349.481380] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.771Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16696da66e06998a20a74d5a","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16696da66e06998a20a74d5a: start","job_status":"start","queue_duration_s":2.179471,"scheduling_latency_s":2.179488}1518gitlab # [ 349.494076] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.788Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16696da66e06998a20a74d5a","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16696da66e06998a20a74d5a: done: 0.017222 sec","job_status":"done","queue_duration_s":2.179471,"scheduling_latency_s":2.179488,"redis_calls":13,"redis_duration_s":0.001587,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000575,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001012,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010057,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017222,"completed_at":"2026-10-02T14:15:05.788Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1519gitlab # [ 349.534950] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.789Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5fda38382c28388219ace264","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5fda38382c28388219ace264: start","job_status":"start","queue_duration_s":2.197863,"scheduling_latency_s":2.197881}1520gitlab # [ 349.548117] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.805Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5fda38382c28388219ace264","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5fda38382c28388219ace264: done: 0.016092 sec","job_status":"done","queue_duration_s":2.197863,"scheduling_latency_s":2.197881,"redis_calls":13,"redis_duration_s":0.001676,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000599,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001077,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011456,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016092,"completed_at":"2026-10-02T14:15:05.805Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1521gitlab # [ 349.589923] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.807Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"52143cee5df5eac8024db4c0","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-52143cee5df5eac8024db4c0: start","job_status":"start","queue_duration_s":2.215146,"scheduling_latency_s":2.215163}1522gitlab # [ 349.602923] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.823Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"52143cee5df5eac8024db4c0","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-52143cee5df5eac8024db4c0: done: 0.016216 sec","job_status":"done","queue_duration_s":2.215146,"scheduling_latency_s":2.215163,"redis_calls":13,"redis_duration_s":0.0018969999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000646,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001251,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011394,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016216,"completed_at":"2026-10-02T14:15:05.823Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1523gitlab # [ 349.644043] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.824Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f8dd7b6b2acacd82fc24ccd","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f8dd7b6b2acacd82fc24ccd: start","job_status":"start","queue_duration_s":2.232521,"scheduling_latency_s":2.232538}1524gitlab # [ 349.656883] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.840Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f8dd7b6b2acacd82fc24ccd","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f8dd7b6b2acacd82fc24ccd: done: 0.015579 sec","job_status":"done","queue_duration_s":2.232521,"scheduling_latency_s":2.232538,"redis_calls":13,"redis_duration_s":0.0015729999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000569,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001004,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010131,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015579,"completed_at":"2026-10-02T14:15:05.839Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1525gitlab # [ 349.697779] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.841Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6eac5733b18d8500a33b7ac2","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6eac5733b18d8500a33b7ac2: start","job_status":"start","queue_duration_s":2.249953,"scheduling_latency_s":2.24997}1526gitlab # [ 349.710769] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.859Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6eac5733b18d8500a33b7ac2","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6eac5733b18d8500a33b7ac2: done: 0.017275 sec","job_status":"done","queue_duration_s":2.249953,"scheduling_latency_s":2.24997,"redis_calls":15,"redis_duration_s":0.0019,"redis_read_bytes":266,"redis_write_bytes":1277,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000253,"redis_feature_flag_read_bytes":256,"redis_feature_flag_write_bytes":120,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000629,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001018,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012488,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017275,"completed_at":"2026-10-02T14:15:05.859Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1527gitlab # [ 349.753527] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.860Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"89a0be1fb6c0da79501794a6","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-89a0be1fb6c0da79501794a6: start","job_status":"start","queue_duration_s":2.268326,"scheduling_latency_s":2.268343}1528gitlab # [ 349.766456] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.876Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"89a0be1fb6c0da79501794a6","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-89a0be1fb6c0da79501794a6: done: 0.016261 sec","job_status":"done","queue_duration_s":2.268326,"scheduling_latency_s":2.268343,"redis_calls":14,"redis_duration_s":0.0017770000000000002,"redis_read_bytes":138,"redis_write_bytes":1221,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000164,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":64,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00058,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001033,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011614,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016261,"completed_at":"2026-10-02T14:15:05.876Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1529gitlab # [ 349.810182] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.877Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"47184aeb07f86f16ab239947","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-47184aeb07f86f16ab239947: start","job_status":"start","queue_duration_s":2.285693,"scheduling_latency_s":2.285709}1530gitlab # [ 349.823165] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.894Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"47184aeb07f86f16ab239947","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-47184aeb07f86f16ab239947: done: 0.016501 sec","job_status":"done","queue_duration_s":2.285693,"scheduling_latency_s":2.285709,"redis_calls":14,"redis_duration_s":0.001799,"redis_read_bytes":138,"redis_write_bytes":1269,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000168,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":112,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000584,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001047,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011765,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016501,"completed_at":"2026-10-02T14:15:05.894Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1531gitlab # [ 349.866397] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.896Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4aaebd0acfb6725fbd54e924","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4aaebd0acfb6725fbd54e924: start","job_status":"start","queue_duration_s":2.304364,"scheduling_latency_s":2.304385}1532gitlab # [ 349.879430] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.913Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4aaebd0acfb6725fbd54e924","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4aaebd0acfb6725fbd54e924: done: 0.017278 sec","job_status":"done","queue_duration_s":2.304364,"scheduling_latency_s":2.304385,"redis_calls":13,"redis_duration_s":0.0016489999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000573,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001076,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011784,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017278,"completed_at":"2026-10-02T14:15:05.913Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1533gitlab # [ 349.920748] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.914Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"47eed88b552d530cf72c50f3","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-47eed88b552d530cf72c50f3: start","job_status":"start","queue_duration_s":2.322851,"scheduling_latency_s":2.322867}1534gitlab # [ 349.933606] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.931Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"47eed88b552d530cf72c50f3","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-47eed88b552d530cf72c50f3: done: 0.016558 sec","job_status":"done","queue_duration_s":2.322851,"scheduling_latency_s":2.322867,"redis_calls":13,"redis_duration_s":0.001702,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000577,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001125,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011734,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016558,"completed_at":"2026-10-02T14:15:05.931Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1535gitlab # [ 349.974119] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.932Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2fd1e2b1df35c76d17454922","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2fd1e2b1df35c76d17454922: start","job_status":"start","queue_duration_s":2.340552,"scheduling_latency_s":2.340568}1536gitlab # [ 349.988095] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.953Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2fd1e2b1df35c76d17454922","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2fd1e2b1df35c76d17454922: done: 0.021376 sec","job_status":"done","queue_duration_s":2.340552,"scheduling_latency_s":2.340568,"redis_calls":13,"redis_duration_s":0.0016359999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000612,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001024,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016779,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021376,"completed_at":"2026-10-02T14:15:05.953Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1537gitlab # [ 350.029264] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.955Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"57a381ac3419b7f3e58beb8a","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-57a381ac3419b7f3e58beb8a: start","job_status":"start","queue_duration_s":2.36313,"scheduling_latency_s":2.363147}1538gitlab # [ 350.042319] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.970Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"57a381ac3419b7f3e58beb8a","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-57a381ac3419b7f3e58beb8a: done: 0.015784 sec","job_status":"done","queue_duration_s":2.36313,"scheduling_latency_s":2.363147,"redis_calls":13,"redis_duration_s":0.001597,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000578,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001019,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010112,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015784,"completed_at":"2026-10-02T14:15:05.970Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1539gitlab # [ 350.083698] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.973Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16a2238053dd596f38d691c4","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16a2238053dd596f38d691c4: start","job_status":"start","queue_duration_s":2.381245,"scheduling_latency_s":2.381262}1540gitlab # [ 350.096964] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.989Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16a2238053dd596f38d691c4","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16a2238053dd596f38d691c4: done: 0.016016 sec","job_status":"done","queue_duration_s":2.381245,"scheduling_latency_s":2.381262,"redis_calls":13,"redis_duration_s":0.00167,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000631,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001039,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011323,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016016,"completed_at":"2026-10-02T14:15:05.989Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1541gitlab # [ 350.137981] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:05.990Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"307f853156d52da5efd5f971","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-307f853156d52da5efd5f971: start","job_status":"start","queue_duration_s":2.398541,"scheduling_latency_s":2.398559}1542gitlab # [ 350.151074] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:06.006Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"307f853156d52da5efd5f971","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-307f853156d52da5efd5f971: done: 0.015998 sec","job_status":"done","queue_duration_s":2.398541,"scheduling_latency_s":2.398559,"redis_calls":13,"redis_duration_s":0.001681,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000575,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001106,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011274,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015998,"completed_at":"2026-10-02T14:15:06.006Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1543gitlab # [ 350.193449] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:06.007Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f7c50278d266252ca513f568","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f7c50278d266252ca513f568: start","job_status":"start","queue_duration_s":2.415704,"scheduling_latency_s":2.41572}1544gitlab # [ 350.206443] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:06.023Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f7c50278d266252ca513f568","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f7c50278d266252ca513f568: done: 0.016309 sec","job_status":"done","queue_duration_s":2.415704,"scheduling_latency_s":2.41572,"redis_calls":13,"redis_duration_s":0.0016519999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000589,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001063,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011541,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016309,"completed_at":"2026-10-02T14:15:06.023Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1545gitlab # [ 350.247764] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:06.025Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fd55d41db641cadf18055c3d","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fd55d41db641cadf18055c3d: start","job_status":"start","queue_duration_s":2.433131,"scheduling_latency_s":2.433147}1546gitlab # [ 350.260994] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:06.040Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fd55d41db641cadf18055c3d","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fd55d41db641cadf18055c3d: done: 0.015481 sec","job_status":"done","queue_duration_s":2.433131,"scheduling_latency_s":2.433147,"redis_calls":13,"redis_duration_s":0.001616,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000562,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001054,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009979,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015481,"completed_at":"2026-10-02T14:15:06.040Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1547gitlab # [ 350.301871] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:06.042Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e00343cab2530cf999fc3a7d","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e00343cab2530cf999fc3a7d: start","job_status":"start","queue_duration_s":2.450826,"scheduling_latency_s":2.450843}1548gitlab # [ 350.314957] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:06.058Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e00343cab2530cf999fc3a7d","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e00343cab2530cf999fc3a7d: done: 0.015884 sec","job_status":"done","queue_duration_s":2.450826,"scheduling_latency_s":2.450843,"redis_calls":13,"redis_duration_s":0.002537,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001004,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011247,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015884,"completed_at":"2026-10-02T14:15:06.058Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1549gitlab # [ 350.356138] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:06.059Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e2577542d791515f40b5d101","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e2577542d791515f40b5d101: start","job_status":"start","queue_duration_s":2.467832,"scheduling_latency_s":2.467849}1550gitlab # [ 350.369088] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:06.075Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e2577542d791515f40b5d101","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e2577542d791515f40b5d101: done: 0.015709 sec","job_status":"done","queue_duration_s":2.467832,"scheduling_latency_s":2.467849,"redis_calls":13,"redis_duration_s":0.001616,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000564,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001052,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011113,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015709,"completed_at":"2026-10-02T14:15:06.075Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1551gitlab # [ 350.410963] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:06.076Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb08dfa39bddcf0ecdec7289","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb08dfa39bddcf0ecdec7289: start","job_status":"start","queue_duration_s":2.484676,"scheduling_latency_s":2.484692}1552gitlab # [ 350.424101] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:06.092Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb08dfa39bddcf0ecdec7289","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb08dfa39bddcf0ecdec7289: done: 0.015704 sec","job_status":"done","queue_duration_s":2.484676,"scheduling_latency_s":2.484692,"redis_calls":13,"redis_duration_s":0.001634,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000561,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001073,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011163,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015704,"completed_at":"2026-10-02T14:15:06.092Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1553gitlab # [ 350.466804] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:06.093Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef09397dd9e9e621bd22a30d","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef09397dd9e9e621bd22a30d: start","job_status":"start","queue_duration_s":2.50145,"scheduling_latency_s":2.501466}1554gitlab # [ 350.480424] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:06.109Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef09397dd9e9e621bd22a30d","created_at":"2026-10-02T14:15:03.437Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-4a56cfe134eb49b9","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"88261a1da4e12de832958c7ef08f4843","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:03.591Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef09397dd9e9e621bd22a30d: done: 0.015633 sec","job_status":"done","queue_duration_s":2.50145,"scheduling_latency_s":2.501466,"redis_calls":13,"redis_duration_s":0.001588,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00057,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001018,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009965,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015633,"completed_at":"2026-10-02T14:15:06.108Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1555gitlab # [ 350.522861] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:06.111Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"11e63541bc458b4b10089c67","created_at":"2026-10-02T14:15:05.056Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-ea524fda095f4a82","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"33b702ed99e0de2b3c4e8ab2f979d533","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:05.095Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-11e63541bc458b4b10089c67: start","job_status":"start","queue_duration_s":1.015927,"scheduling_latency_s":1.015943}1556gitlab # [ 350.536364] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:06.128Z","message":"No existing merge request to be cleaned up."}1557gitlab # [ 350.538173] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:06.138Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"11e63541bc458b4b10089c67","created_at":"2026-10-02T14:15:05.056Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-ea524fda095f4a82","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"33b702ed99e0de2b3c4e8ab2f979d533","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:05.095Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-11e63541bc458b4b10089c67: done: 0.026795 sec","job_status":"done","queue_duration_s":1.015927,"scheduling_latency_s":1.015943,"redis_calls":10,"redis_duration_s":0.001502,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000653,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000849,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015616,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026795,"completed_at":"2026-10-02T14:15:06.138Z","load_balancing_strategy":"primary","db_duration_s":0.012519,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1558gitlab # [ 350.580887] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:06.139Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d9a8d5f95654aa4c631e95a3","created_at":"2026-10-02T14:15:05.056Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-ea524fda095f4a82","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"33b702ed99e0de2b3c4e8ab2f979d533","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:05.095Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d9a8d5f95654aa4c631e95a3: start","job_status":"start","queue_duration_s":1.044014,"scheduling_latency_s":1.044031}1559gitlab # [ 350.594744] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:06.151Z","message":"No existing merge request to be cleaned up."}1560gitlab # [ 350.596723] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:06.159Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d9a8d5f95654aa4c631e95a3","created_at":"2026-10-02T14:15:05.056Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-ea524fda095f4a82","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"33b702ed99e0de2b3c4e8ab2f979d533","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:05.095Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d9a8d5f95654aa4c631e95a3: done: 0.019682 sec","job_status":"done","queue_duration_s":1.044014,"scheduling_latency_s":1.044031,"redis_calls":10,"redis_duration_s":0.001498,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000637,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000861,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014459,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019682,"completed_at":"2026-10-02T14:15:06.159Z","load_balancing_strategy":"primary","db_duration_s":0.001419,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1561gitlab # [ 350.639766] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:06.160Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"57ae950c83e9b72e92ae8085","created_at":"2026-10-02T14:15:05.056Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-ea524fda095f4a82","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"33b702ed99e0de2b3c4e8ab2f979d533","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:05.095Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-57ae950c83e9b72e92ae8085: start","job_status":"start","queue_duration_s":1.064813,"scheduling_latency_s":1.064831}1562gitlab # [ 350.653341] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:06.171Z","message":"No existing merge request to be cleaned up."}1563gitlab # [ 350.655209] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:06.181Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"57ae950c83e9b72e92ae8085","created_at":"2026-10-02T14:15:05.056Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-ea524fda095f4a82","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"33b702ed99e0de2b3c4e8ab2f979d533","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:05.095Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-57ae950c83e9b72e92ae8085: done: 0.0208 sec","job_status":"done","queue_duration_s":1.064813,"scheduling_latency_s":1.064831,"redis_calls":10,"redis_duration_s":0.001485,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000662,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000823,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014371,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0208,"completed_at":"2026-10-02T14:15:06.181Z","load_balancing_strategy":"primary","db_duration_s":0.00145,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1564gitlab # [ 350.697227] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:06.182Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d55a5bd3e31e7183c2b0a64d","created_at":"2026-10-02T14:15:05.056Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-ea524fda095f4a82","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"33b702ed99e0de2b3c4e8ab2f979d533","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:05.095Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d55a5bd3e31e7183c2b0a64d: start","job_status":"start","queue_duration_s":1.08675,"scheduling_latency_s":1.086767}1565gitlab # [ 350.710592] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:06.193Z","message":"No existing merge request to be cleaned up."}1566gitlab # [ 350.712445] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:06.201Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d55a5bd3e31e7183c2b0a64d","created_at":"2026-10-02T14:15:05.056Z","trace_propagation_headers":{"sentry-trace":"e324910c3ed14400842a05a567a8aa83-ea524fda095f4a82","baggage":"sentry-trace_id=e324910c3ed14400842a05a567a8aa83,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"33b702ed99e0de2b3c4e8ab2f979d533","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:15:05.095Z","job_size_bytes":2,"pid":2132,"sidekiq_tid":"m7mw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d55a5bd3e31e7183c2b0a64d: done: 0.019617 sec","job_status":"done","queue_duration_s":1.08675,"scheduling_latency_s":1.086767,"redis_calls":10,"redis_duration_s":0.001488,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000614,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000874,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014407,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019617,"completed_at":"2026-10-02T14:15:06.201Z","load_balancing_strategy":"primary","db_duration_s":0.001382,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1567gitlab # [ 391.195678] postgres[3856]: [3856] ERROR: relation "table_without_models" does not exist1568gitlab # [ 391.196170] postgres[3856]: [3856] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1569gitlab # [ 397.031396] runuser[3392]: pam_unix(runuser:session): session closed for user gitlab1570gitlab # [ 397.032761] setup-gitlab-start[3354]: + systemctl restart gitlab-sidekiq.service1571gitlab # [ 397.059086] systemd[1]: Stopping gitlab-sidekiq.service...1572gitlab # [ 397.063640] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:54.969Z","message":"Shutting down"}1573gitlab # [ 397.066391] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:54.972Z","message":"Scheduler exiting..."}1574gitlab # [ 397.069007] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:54.977Z","message":"Terminating quiet threads for default capsule"}1575gitlab # [ 397.072099] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:54.981Z","message":"Scheduler exiting..."}1576gitlab # [ 397.576622] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:55.486Z","message":"Pausing to allow jobs to finish..."}1577gitlab # [ 399.081537] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:56.990Z","message":"Bye!"}1578gitlab # [ 399.119220] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-10-02T14:15:57.029Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":2132,"worker_id":"sidekiq_0","memwd_rss_bytes":960548864,"retry":0}1579gitlab # [ 399.997582] systemd[1]: gitlab-sidekiq.service: Deactivated successfully.1580gitlab # [ 399.997932] systemd[1]: Stopped gitlab-sidekiq.service.1581gitlab # [ 399.998256] systemd[1]: gitlab-sidekiq.service: Consumed 2min 40.245s CPU time over 4min 31.377s wall clock time, 945.8M memory peak, 4.3M read from disk.1582gitlab # [ 400.005840] systemd[1]: Started gitlab-sidekiq.service.1583gitlab # [ 400.009342] setup-gitlab-start[3354]: + mkdir -p /var/lib/secrets1584gitlab # [ 400.012466] setup-gitlab-start[3354]: + echo nixbot-test-token-of-20-chars1585gitlab # [ 400.014098] setup-gitlab-start[3354]: + chmod 644 /var/lib/secrets/gitlab-token1586gitlab # [ 400.016364] setup-gitlab-start[3354]: + timeout 300 bash -c $'until curl -fs -H "PRIVATE-TOKEN: $0" http://gitlab/api/v4/projects/root%2Ftest-flake > /dev/null; do\n curl -s -X POST -H "PRIVATE-TOKEN: $0" -H "Content-Type: application/json" \\\n -d "{\\"name\\":\\"test-flake\\",\\"visibility\\":\\"public\\",\\"topics\\":[\\"build-with-buildbot\\"]}" \\\n http://gitlab/api/v4/projects | jq -c "{id, message, error}"\n sleep 5\ndone' nixbot-test-token-of-20-chars1587gitlab # [ 401.253687] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:15:59 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 404 35 "" "curl/8.21.0" 12081588gitlab # [ 402.196744] gitaly[930]: time="2026-10-02T14:16:00.105Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFHKECRW6ANSJRVVH71AED error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T14:16:10.005" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:16:00.105" pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1589gitlab # [ 402.197124] gitaly[930]: time="2026-10-02T14:16:00.106Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFHKECRW6ANSJRVVH71AED grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T14:16:10.005" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:16:00.105" grpc.time_ms=0.441 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1590gitlab # [ 402.369499] gitaly[930]: time="2026-10-02T14:16:00.279Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFHKECRW6ANSJRVVH71AED error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T14:16:10.178" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:16:00.279" pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1591gitlab # [ 402.386980] gitaly[930]: time="2026-10-02T14:16:00.279Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFHKECRW6ANSJRVVH71AED grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T14:16:10.178" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:16:00.279" grpc.time_ms=0.402 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1592gitlab # [ 402.415642] gitaly[930]: time="2026-10-02T14:16:00.325Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFHKECRW6ANSJRVVH71AED diskcache=c1fc01fb-5749-480b-a957-2f18c1cfc3d8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-02T14:16:10.186" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:16:00.287" pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1593gitlab # [ 402.433542] gitaly[930]: time="2026-10-02T14:16:00.326Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=17 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1438 command.oublock=32 command.real_time_ms=20 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFHKECRW6ANSJRVVH71AED grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-02T14:16:10.186" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:16:00.287" grpc.time_ms=38.469 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1594gitlab # [ 402.456812] gitaly[930]: time="2026-10-02T14:16:00.336Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFHKECRW6ANSJRVVH71AED grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T14:16:10.235" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:16:00.336" pid=930 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1595gitlab # [ 402.474334] gitaly[930]: time="2026-10-02T14:16:00.336Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFHKECRW6ANSJRVVH71AED grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T14:16:10.235" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:16:00.336" grpc.time_ms=0.685 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1596gitlab # [ 402.491534] gitaly[930]: time="2026-10-02T14:16:00.373Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=30 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFHKECRW6ANSJRVVH71AED grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-02T14:16:10.240" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:16:00.340" grpc.time_ms=32.179 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1597gitlab # [ 402.552797] gitaly[930]: time="2026-10-02T14:16:00.462Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFHKECRW6ANSJRVVH71AED error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T14:16:10.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:16:00.462" pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1598gitlab # [ 402.570332] gitaly[930]: time="2026-10-02T14:16:00.463Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFHKECRW6ANSJRVVH71AED grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T14:16:10.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:16:00.462" grpc.time_ms=0.422 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1599gitlab # [ 402.598203] gitaly[930]: time="2026-10-02T14:16:00.508Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFHKECRW6ANSJRVVH71AED diskcache=2eef7e08-a5cf-4313-8a15-ae0e90456eea grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-02T14:16:10.005" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:16:00.465" pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1600gitlab # [ 402.616416] gitaly[930]: time="2026-10-02T14:16:00.508Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=22 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1439 command.oublock=32 command.real_time_ms=26 command.spawn_token_fork_ms=1 command.system_time_ms=22 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFHKECRW6ANSJRVVH71AED grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-02T14:16:10.005" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:16:00.465" grpc.time_ms=42.901 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1601gitlab # [ 402.636823] gitaly[930]: time="2026-10-02T14:16:00.517Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFHKECRW6ANSJRVVH71AED grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T14:16:10.055" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:16:00.516" pid=930 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1602gitlab # [ 402.649725] gitaly[930]: time="2026-10-02T14:16:00.517Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFHKECRW6ANSJRVVH71AED grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T14:16:10.055" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:16:00.516" grpc.time_ms=1.069 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1603gitlab # [ 402.662543] gitaly[930]: time="2026-10-02T14:16:00.534Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFHKECRW6ANSJRVVH71AED grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-02T14:16:10.061" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:16:00.522" grpc.time_ms=12.5 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1604gitlab # [ 402.678264] gitaly[930]: time="2026-10-02T14:16:00.578Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFHKECRW6ANSJRVVH71AED grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T14:16:10.116" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:16:00.577" grpc.time_ms=0.371 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1605gitlab # [ 402.690702] gitaly[930]: time="2026-10-02T14:16:00.588Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFHKECRW6ANSJRVVH71AED grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-02T14:16:10.127" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:16:00.588" grpc.time_ms=0.58 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1606gitlab # [ 403.419486] gitaly[930]: time="2026-10-02T14:16:01.328Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFHKECRW6ANSJRVVH71AED grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-02T14:16:11.210" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:16:01.312" grpc.time_ms=15.939 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1607gitlab # [ 404.139297] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:02 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 28671608gitlab # [ 404.145796] setup-gitlab-start[3929]: {"id":1,"message":null,"error":null}1609gitlab # [ 407.901491] sidekiq-cluster[3911]: {"severity":"INFO","time":"2026-10-02T14:16:05.769Z","message":"Starting cluster with 1 processes"}1610gitlab # [ 409.944933] gitaly[930]: time="2026-10-02T14:16:07.854Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=4 command.system_time_ms=5 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFHV5DNS3CFN2ZWWYNZ79A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-02T14:16:17.009" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:16:07.840" grpc.time_ms=13.653 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1611gitlab # [ 413.480482] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:11 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 200 4924 "" "curl/8.21.0" 43031612gitlab # [ 413.484871] setup-gitlab-start[3354]: + mkdir -p /tmp/test-flake1613gitlab # [ 413.487239] setup-gitlab-start[3354]: + cd /tmp/test-flake1614gitlab # [ 413.488936] setup-gitlab-start[3354]: + git init -b master1615gitlab # [ 413.503775] setup-gitlab-start[4032]: Initialized empty Git repository in /tmp/test-flake/.git/1616gitlab # [ 413.505981] setup-gitlab-start[3354]: + git config user.name test1617gitlab # [ 413.507823] setup-gitlab-start[3354]: + git config user.email test@example.com1618gitlab # [ 413.512930] setup-gitlab-start[3354]: + cat1619gitlab # [ 413.516973] setup-gitlab-start[3354]: + git add flake.nix1620gitlab # [ 413.522533] setup-gitlab-start[3354]: + git commit -m 'initial commit'1621gitlab # [ 413.542674] setup-gitlab-start[4037]: [master (root-commit) eb0e348] initial commit1622gitlab # [ 413.544630] setup-gitlab-start[4037]: 1 file changed, 10 insertions(+)1623gitlab # [ 413.546147] setup-gitlab-start[4037]: create mode 100644 flake.nix1624gitlab # [ 413.547643] setup-gitlab-start[3354]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git1625gitlab # [ 413.550434] setup-gitlab-start[3354]: + git push -u origin master1626gitlab # [ 413.661178] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:11 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 821627gitlab # [ 413.797763] gitaly[930]: time="2026-10-02T14:16:11.707Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFHZHM7KYC7Q2ETVQWR5A0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:11.695" grpc.time_ms=11.634 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1628gitlab # [ 413.820893] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:11 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 1351629gitlab # [ 414.111549] gitaly[930]: time="2026-10-02T14:16:12.021Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=570 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFHZP98CM7CNP4Q3XMNH64 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-02T14:16:22.008" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:16:12.009" grpc.time_ms=11.892 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1630gitlab # [ 414.174025] gitaly[930]: time="2026-10-02T14:16:12.084Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=302 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M3YFHZP98CM7CNP4Q3XMNH64 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-10-02T14:16:42.065" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-10-02T14:16:12.066" grpc.time_ms=17.81 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1631gitlab # [ 414.207345] gitaly[930]: time="2026-10-02T14:16:12.117Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M3YFHZP98CM7CNP4Q3XMNH64 duration_ms=202ns method=POST pid=930 status=200 url="http://unix/api/v4/internal/allowed"1632gitlab # [ 414.250146] gitaly[930]: time="2026-10-02T14:16:12.160Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M3YFHZP98CM7CNP4Q3XMNH64 duration_ms=41ns method=POST pid=930 status=200 url="http://unix/api/v4/internal/pre_receive"1633gitlab # [ 414.258008] gitaly[930]: time="2026-10-02T14:16:12.160Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFHZP98CM7CNP4Q3XMNH64 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5316 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:16:11.913" grpc.time_ms=247.195 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1634gitlab # [ 414.290565] gitaly[930]: time="2026-10-02T14:16:12.200Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFHZP98CM7CNP4Q3XMNH64 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4786 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:16:12.199" grpc.time_ms=0.899 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1635gitlab # [ 414.322920] gitaly[930]: time="2026-10-02T14:16:12.233Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFHZP98CM7CNP4Q3XMNH64 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4785 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:16:12.232" grpc.time_ms=0.622 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1636gitlab # [ 414.376745] gitaly[930]: time="2026-10-02T14:16:12.286Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFHZP98CM7CNP4Q3XMNH64 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:16:12.285" grpc.time_ms=0.902 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1637gitlab # [ 414.498000] gitaly[930]: time="2026-10-02T14:16:12.408Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFHZP98CM7CNP4Q3XMNH64 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-02T14:16:22.295" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:16:12.395" grpc.time_ms=12.3 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1638gitlab # [ 414.559365] gitaly[930]: time="2026-10-02T14:16:12.469Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=23 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1436 command.oublock=0 command.real_time_ms=25 command.spawn_token_fork_ms=1 command.system_time_ms=16 command.user_time_ms=7 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFHZP98CM7CNP4Q3XMNH64 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-10-02T14:16:22.007" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:16:12.438" grpc.time_ms=30.509 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1639gitlab # [ 414.582279] gitaly[930]: time="2026-10-02T14:16:12.485Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M3YFHZP98CM7CNP4Q3XMNH64 duration_ms=163ns method=POST pid=930 status=200 url="http://unix/api/v4/internal/post_receive"1640gitlab # [ 414.587027] gitaly[930]: time="2026-10-02T14:16:12.486Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFHZP98CM7CNP4Q3XMNH64 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4810 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:16:12.320" grpc.time_ms=165.835 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1641gitlab # [ 414.602992] gitaly[930]: time="2026-10-02T14:16:12.498Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M3YFHZP98CM7CNP4Q3XMNH64 diskcache=83512cd0-334c-4334-91fa-515f88913f3c grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:11.842" pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1642gitlab # [ 414.617916] setup-gitlab-start[4041]: To http://gitlab/root/test-flake.git1643gitlab # [ 414.619285] setup-gitlab-start[4041]: * [new branch] master -> master1644gitlab # [ 414.620658] setup-gitlab-start[4041]: branch 'master' set up to track 'origin/master'.1645gitlab # [ 414.622781] gitlab-workhorse[816]: time="2026-10-02T14:16:12Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521646gitlab # [ 414.625422] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:12 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 7771647gitlab # [ 414.628385] systemd[1]: Finished setup-gitlab.service.1648gitlab # [ 414.629391] gitaly[930]: time="2026-10-02T14:16:12.498Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=230 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=847872 command.minflt=13468 command.oublock=32 command.real_time_ms=649 command.spawn_token_fork_ms=0 command.system_time_ms=132 command.user_time_ms=98 component=gitaly.StreamServerInterceptor correlation_id=01M3YFHZP98CM7CNP4Q3XMNH64 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:11.842" grpc.time_ms=656.294 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1649gitlab # [ 414.645533] systemd[1]: Starting nixbot CI service...1650gitlab: (finished: waiting for unit setup-gitlab.service, in 415.79 seconds)1651gitlab: waiting for unit nixbot.service1652gitlab # [ 416.041667] nixbot[4119]: {"ts": "2026-10-02T14:16:13.950Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1653gitlab # [ 416.088979] nixbot[4119]: {"ts": "2026-10-02T14:16:13.998Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1654gitlab # [ 416.118323] nixbot[4119]: {"ts": "2026-10-02T14:16:14.028Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1655gitlab # [ 416.124340] nixbot[4119]: {"ts": "2026-10-02T14:16:14.034Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1656gitlab # [ 416.129387] nixbot[4119]: {"ts": "2026-10-02T14:16:14.039Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1657gitlab # [ 416.137997] nixbot[4119]: {"ts": "2026-10-02T14:16:14.047Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1658gitlab # [ 416.143332] nixbot[4119]: {"ts": "2026-10-02T14:16:14.053Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1659gitlab # [ 416.147460] nixbot[4119]: {"ts": "2026-10-02T14:16:14.055Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1660gitlab # [ 416.153473] nixbot[4119]: {"ts": "2026-10-02T14:16:14.063Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1661gitlab # [ 416.157189] nixbot[4119]: {"ts": "2026-10-02T14:16:14.066Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1662gitlab # [ 416.162963] nixbot[4119]: {"ts": "2026-10-02T14:16:14.072Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1663gitlab # [ 416.170503] nixbot[4119]: {"ts": "2026-10-02T14:16:14.080Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1664gitlab # [ 416.174609] nixbot[4119]: {"ts": "2026-10-02T14:16:14.084Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1665gitlab # [ 416.179319] nixbot[4119]: {"ts": "2026-10-02T14:16:14.086Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1666gitlab # [ 416.183204] nixbot[4119]: {"ts": "2026-10-02T14:16:14.091Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1667gitlab # [ 416.187053] nixbot[4119]: {"ts": "2026-10-02T14:16:14.094Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1668gitlab # [ 416.190643] nixbot[4119]: {"ts": "2026-10-02T14:16:14.096Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1669gitlab # [ 416.194384] nixbot[4119]: {"ts": "2026-10-02T14:16:14.097Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1670gitlab # [ 416.197978] nixbot[4119]: {"ts": "2026-10-02T14:16:14.103Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"}1671gitlab # [ 416.200979] nixbot[4119]: {"ts": "2026-10-02T14:16:14.105Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"}1672gitlab # [ 416.203970] nixbot[4119]: {"ts": "2026-10-02T14:16:14.107Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"}1673gitlab # [ 416.207026] nixbot[4119]: {"ts": "2026-10-02T14:16:14.114Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"}1674gitlab # [ 416.210535] nixbot[4119]: {"ts": "2026-10-02T14:16:14.117Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"}1675gitlab # [ 416.213834] nixbot[4119]: {"ts": "2026-10-02T14:16:14.120Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"}1676gitlab # [ 416.217084] nixbot[4119]: {"ts": "2026-10-02T14:16:14.122Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"}1677gitlab # [ 416.220407] nixbot[4119]: {"ts": "2026-10-02T14:16:14.125Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"}1678gitlab # [ 416.223546] nixbot[4119]: {"ts": "2026-10-02T14:16:14.127Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"}1679gitlab # [ 416.226789] nixbot[4119]: {"ts": "2026-10-02T14:16:14.128Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"}1680gitlab # [ 416.229697] nixbot[4119]: {"ts": "2026-10-02T14:16:14.129Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"}1681gitlab # [ 416.234560] nixbot[4119]: {"ts": "2026-10-02T14:16:14.144Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"}1682gitlab # [ 416.239948] nixbot[4119]: {"ts": "2026-10-02T14:16:14.150Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"}1683gitlab # [ 416.243060] nixbot[4119]: {"ts": "2026-10-02T14:16:14.152Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"}1684gitlab # [ 416.246058] nixbot[4119]: {"ts": "2026-10-02T14:16:14.153Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 32, "migration": "upload_queue"}1685gitlab # [ 416.249095] nixbot[4119]: {"ts": "2026-10-02T14:16:14.156Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 33, "migration": "effect_runs_catchup"}1686gitlab # [ 416.251966] nixbot[4119]: {"ts": "2026-10-02T14:16:14.158Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 34, "migration": "public_badges"}1687gitlab # [ 416.255865] nixbot[4119]: {"ts": "2026-10-02T14:16:14.165Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 35, "migration": "settle_reset_effects"}1688gitlab # [ 416.260264] nixbot[4119]: {"ts": "2026-10-02T14:16:14.170Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 36, "migration": "pr_approvals"}1689gitlab # [ 416.717334] nixbot[4119]: INFO: Started server process [4119]1690gitlab # [ 416.718961] nixbot[4119]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)1691gitlab # [ 416.740881] nixbot[4119]: INFO: ::ffff:127.0.0.1:40296 - "GET /health HTTP/1.1" 200 OK1692gitlab # [ 416.751998] systemd[1]: Started nixbot CI service.1693gitlab # [ 416.753479] systemd[1]: Reached target Multi-User System.1694gitlab # [ 416.756185] systemd[1]: Startup finished in 667ms (kernel) + 3.757s (initrd) + 6min 52.330s (userspace) = 6min 56.755s.1695gitlab # [ 417.351657] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:15 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 6101696gitlab # [ 417.356826] nixbot[4119]: {"ts": "2026-10-02T14:16:15.262Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1697gitlab # [ 417.383021] nixbot[4119]: {"ts": "2026-10-02T14:16:15.292Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1698gitlab: (finished: waiting for unit nixbot.service, in 2.11 seconds)1699gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1700gitlab # [ 417.498522] nixbot[4119]: INFO: ::ffff:127.0.0.1:40298 - "GET /health HTTP/1.1" 200 OK1701gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.04 seconds)1702subtest: project discovered and webhook registered1703gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1704gitlab # [ 417.520499] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:15 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 1271705gitlab # [ 417.524417] nixbot[4119]: {"ts": "2026-10-02T14:16:15.434Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1706gitlab # [ 417.528438] nixbot[4119]: {"ts": "2026-10-02T14:16:15.436Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1707gitlab # [ 417.627189] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:15 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.21.0" 941708gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.13 seconds)1709gitlab # [ 417.781672] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:15 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 917 "" "python-httpx/0.28.1" 2521710gitlab # [ 417.790275] nixbot[4119]: {"ts": "2026-10-02T14:16:15.697Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""}1711gitlab # [ 418.133211] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:16 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 3321712gitlab # [ 418.136389] nixbot[4119]: {"ts": "2026-10-02T14:16:16.044Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1713gitlab # [ 418.254321] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:16 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 880 "" "python-httpx/0.28.1" 1051714gitlab # [ 418.257309] nixbot[4119]: {"ts": "2026-10-02T14:16:16.166Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1715gitlab # [ 418.454369] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:16 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 917 "" "python-httpx/0.28.1" 1901716gitlab # [ 418.459248] nixbot[4119]: {"ts": "2026-10-02T14:16:16.364Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""}1717gitlab # [ 418.582681] gitaly[930]: time="2026-10-02T14:16:16.491Z" level=info msg="finished unary call with code OK" catfile.duration_ms=4 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=4 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=3 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ47K3911BFV3R2Q8JK4H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-10-02T14:16:46.068" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:16:16.469" grpc.time_ms=22.053 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1718gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1719gitlab # [ 418.634101] gitaly[930]: time="2026-10-02T14:16:16.543Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ47K3911BFV3R2Q8JK4H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:16:46.141" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:16.542" grpc.time_ms=1.26 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1720gitlab # [ 418.656941] gitaly[930]: time="2026-10-02T14:16:16.565Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=581 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ47K3911BFV3R2Q8JK4H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-10-02T14:16:26.000" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:16.551" grpc.time_ms=14.302 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1721gitlab # [ 418.740352] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:16 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 2731722gitlab # [ 418.744465] nixbot[4119]: {"ts": "2026-10-02T14:16:16.647Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""}1723gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.17 seconds)1724gitlab # [ 418.799705] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:16 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 880 "" "curl/8.21.0" 931725(finished: subtest: project discovered and webhook registered, in 1.30 seconds)1726subtest: push delivers the webhook1727gitlab: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails'1728gitlab # [ 418.937839] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:16 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 1851729gitlab # [ 418.941004] nixbot[4119]: {"ts": "2026-10-02T14:16:16.849Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""}1730gitlab # [ 418.946160] nixbot[4119]: {"ts": "2026-10-02T14:16:16.856Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1731gitlab # [ 419.104641] gitaly[930]: time="2026-10-02T14:16:17.014Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=577 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M3YFJ4RJWDKTB01TKRNSFE03 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:17.003" grpc.time_ms=11.328 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc1732gitlab # [ 419.119419] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:17 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1011733gitlab # [ 419.272251] gitaly[930]: time="2026-10-02T14:16:17.182Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ4VT2AY7ZBDBEXV1E101 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:17.169" pid=930 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc1734gitlab # [ 419.288104] gitaly[930]: time="2026-10-02T14:16:17.182Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ4VT2AY7ZBDBEXV1E101 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:17.169" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc1735gitlab # [ 419.306387] gitlab-workhorse[816]: time="2026-10-02T14:16:17Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1131736gitlab # [ 419.308841] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:17 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 1651737gitlab # [ 419.311166] gitaly[930]: time="2026-10-02T14:16:17.185Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ4VT2AY7ZBDBEXV1E101 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:17.169" grpc.time_ms=13.865 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc1738gitlab # [ 419.402609] gitaly[930]: time="2026-10-02T14:16:17.312Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ515BS0QZSNXFNRQHMYZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:17.288" pid=930 remote_ip="2001:db8:1::1" request_sha=40e263a9abba00dc8a6f8c15f23130e1da9fe30a response_bytes=223 span.kind=server system=grpc1739gitlab # [ 419.420784] gitaly[930]: time="2026-10-02T14:16:17.312Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=19 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1136 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=9 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ515BS0QZSNXFNRQHMYZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:17.288" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc1740gitlab # [ 419.443671] gitlab-workhorse[816]: time="2026-10-02T14:16:17Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2231741gitlab # [ 419.446528] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:17 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 223 "" "git/2.55.0" 1241742gitlab # [ 419.449379] gitaly[930]: time="2026-10-02T14:16:17.313Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=19 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1136 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=9 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ515BS0QZSNXFNRQHMYZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:17.288" grpc.time_ms=24.446 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc1743gitlab # [ 419.533427] gitaly[930]: time="2026-10-02T14:16:17.443Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=577 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M3YFJ56GGK0TY71NEW71S4P8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=192 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:17.430" grpc.time_ms=12.59 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc1744gitlab # [ 419.553721] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:17 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 831745gitlab # [ 419.636696] gitaly[930]: time="2026-10-02T14:16:17.546Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ596FKFS3QHG157MEGHH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:17.533" pid=930 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1746gitlab # [ 419.654627] gitaly[930]: time="2026-10-02T14:16:17.546Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ596FKFS3QHG157MEGHH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:17.533" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc1747gitlab # [ 419.676700] gitlab-workhorse[816]: time="2026-10-02T14:16:17Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671748gitlab # [ 419.679645] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:17 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1001749gitlab # [ 419.682651] gitaly[930]: time="2026-10-02T14:16:17.546Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ596FKFS3QHG157MEGHH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:17.533" grpc.time_ms=13.212 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc1750gitlab # [ 419.761150] gitaly[930]: time="2026-10-02T14:16:17.671Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M3YFJ5DX9ABCS2QNYBH9W9V3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:17.659" grpc.time_ms=11.855 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc1751gitlab # [ 419.782988] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:17 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 731752gitlab # [ 419.844380] gitaly[930]: time="2026-10-02T14:16:17.754Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ5GABHPQMY8QKY775EPS grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:17.732" pid=930 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc1753gitlab # [ 419.867275] gitaly[930]: time="2026-10-02T14:16:17.754Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1101 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=0 command.system_time_ms=13 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ5GABHPQMY8QKY775EPS grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:17.732" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc1754gitlab # [ 419.889959] gitlab-workhorse[816]: time="2026-10-02T14:16:17Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2091755gitlab # [ 419.892988] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:17 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 801756gitlab # [ 419.895982] gitaly[930]: time="2026-10-02T14:16:17.754Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1101 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=0 command.system_time_ms=13 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ5GABHPQMY8QKY775EPS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:17.732" grpc.time_ms=22.303 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc1757gitlab # [ 419.976437] gitaly[930]: time="2026-10-02T14:16:17.886Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ5MGM505KDHE6MBK5E16 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:16:47.484" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:17.885" grpc.time_ms=0.805 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1758gitlab # [ 420.049390] gitaly[930]: time="2026-10-02T14:16:17.959Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=588 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M3YFJ5MGM505KDHE6MBK5E16 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-10-02T14:16:47.545" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:16:17.945" grpc.time_ms=13.312 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1759gitlab # [ 420.087304] gitaly[930]: time="2026-10-02T14:16:17.997Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=4 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M3YFJ5MGM505KDHE6MBK5E16 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T14:16:27.154" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:16:17.985" grpc.time_ms=12.046 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1760gitlab # [ 420.114306] gitaly[930]: time="2026-10-02T14:16:18.024Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFJ5MGM505KDHE6MBK5E16 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T14:16:28.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:16:18.012" grpc.time_ms=11.83 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1761gitlab # [ 420.232118] gitaly[930]: time="2026-10-02T14:16:18.142Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ5MGM505KDHE6MBK5E16 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-02T14:16:28.130" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:16:18.141" grpc.time_ms=0.455 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1762gitlab # [ 420.256476] gitaly[930]: time="2026-10-02T14:16:18.166Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=5 command.system_time_ms=7 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ5MGM505KDHE6MBK5E16 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-02T14:16:28.142" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:16:18.152" grpc.time_ms=13.292 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1763gitlab # [ 420.313229] gitaly[930]: time="2026-10-02T14:16:18.223Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFJ5MGM505KDHE6MBK5E16 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4154 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:16:18.222" grpc.time_ms=0.663 pid=930 span.kind=server system=grpc1764gitlab # [ 420.354548] gitaly[930]: time="2026-10-02T14:16:18.264Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFJ5MGM505KDHE6MBK5E16 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4156 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:16:18.263" grpc.time_ms=0.905 pid=930 span.kind=server system=grpc1765gitlab # [ 420.380261] gitaly[930]: time="2026-10-02T14:16:18.290Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ5MGM505KDHE6MBK5E16 diskcache=40f20c3e-6b23-480f-9a12-cbc4860bcc8d grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-02T14:16:28.159" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:16:18.170" pid=930 ref=refs/keep-around/eb0e348b96a80c3e7fb14424c8e4917e98aa1ea8 remote_ip="2001:db8:1::1" revision=eb0e348b96a80c3e7fb14424c8e4917e98aa1ea8 span.kind=server system=grpc username=root1766gitlab # [ 420.399446] gitaly[930]: time="2026-10-02T14:16:18.290Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=93 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5244 command.oublock=8 command.real_time_ms=112 command.spawn_token_fork_ms=1 command.system_time_ms=51 command.user_time_ms=42 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ5MGM505KDHE6MBK5E16 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-02T14:16:28.159" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:16:18.170" grpc.time_ms=120.09 pid=930 ref=refs/keep-around/eb0e348b96a80c3e7fb14424c8e4917e98aa1ea8 remote_ip="2001:db8:1::1" revision=eb0e348b96a80c3e7fb14424c8e4917e98aa1ea8 span.kind=server system=grpc username=root1767gitlab # [ 420.420576] gitaly[930]: time="2026-10-02T14:16:18.297Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ5MGM505KDHE6MBK5E16 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:16:48.096" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:18.296" grpc.time_ms=0.657 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1768gitlab # [ 420.640223] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:18 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/eb0e348b96a80c3e7fb14424c8e4917e98aa1ea8 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 7401769gitlab # [ 420.645480] nixbot[4119]: {"ts": "2026-10-02T14:16:18.550Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/eb0e348b96a80c3e7fb14424c8e4917e98aa1ea8 \"HTTP/1.1 201 Created\""}1770gitlab # [ 420.651070] nixbot[4119]: {"ts": "2026-10-02T14:16:18.561Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-eb0e348b96a8-0"}1771gitlab # [ 420.851301] nixbot[4119]: {"ts": "2026-10-02T14:16:18.761Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-eb0e348b96a8-0", "argv0": "bwrap"}1772gitlab # [ 420.982546] systemd[1]: Started Nix Daemon.1773gitlab # [ 421.092946] nix-daemon[4288]: accepted connection from pid 4279, user nixbot1774gitlab # [ 426.218210] nix-daemon[4288]: accepted connection from pid 4330, user nixbot1775gitlab # [ 426.242491] nix-daemon[4288]: accepted connection from pid 4279, user nixbot1776gitlab # [ 430.673111] nix-daemon[4288]: accepted connection from pid 4279, user nixbot1777gitlab # [ 430.859514] gitaly[930]: time="2026-10-02T14:16:28.768Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJG7TEEMXFQR8ZJZCYJ5B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:16:58.766" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:28.767" grpc.time_ms=1.204 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1778gitlab # [ 431.123416] gitaly[930]: time="2026-10-02T14:16:29.033Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFJG7TEEMXFQR8ZJZCYJ5B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T14:16:39.016" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:16:29.016" grpc.time_ms=16.47 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1779gitlab # [ 431.262848] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:29 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/eb0e348b96a80c3e7fb14424c8e4917e98aa1ea8 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 5051780gitlab # [ 431.268176] nixbot[4119]: {"ts": "2026-10-02T14:16:29.173Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/eb0e348b96a80c3e7fb14424c8e4917e98aa1ea8 \"HTTP/1.1 201 Created\""}1781gitlab # [ 431.328350] gitaly[930]: time="2026-10-02T14:16:29.238Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJGQTS1MMG5BMQFRY9CA3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:16:59.236" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:29.237" grpc.time_ms=0.841 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1782gitlab # [ 431.380417] gitaly[930]: time="2026-10-02T14:16:29.290Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M3YFJGQTS1MMG5BMQFRY9CA3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T14:16:39.180" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:16:29.280" grpc.time_ms=9.277 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1783gitlab # [ 431.449530] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:29 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/eb0e348b96a80c3e7fb14424c8e4917e98aa1ea8 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1811784gitlab # [ 431.454498] nixbot[4119]: {"ts": "2026-10-02T14:16:29.361Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/eb0e348b96a80c3e7fb14424c8e4917e98aa1ea8 \"HTTP/1.1 201 Created\""}1785gitlab # [ 431.537533] gitaly[930]: time="2026-10-02T14:16:29.447Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJGY1BCAGWGHHHD75YQ6G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:16:59.445" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:29.446" grpc.time_ms=0.868 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1786gitlab # [ 431.589466] gitaly[930]: time="2026-10-02T14:16:29.499Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M3YFJGY1BCAGWGHHHD75YQ6G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T14:16:39.027" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:16:29.487" grpc.time_ms=11.594 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1787gitlab # [ 431.661801] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:29 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/eb0e348b96a80c3e7fb14424c8e4917e98aa1ea8 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1931788gitlab # [ 431.665946] nixbot[4119]: {"ts": "2026-10-02T14:16:29.573Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/eb0e348b96a80c3e7fb14424c8e4917e98aa1ea8 \"HTTP/1.1 201 Created\""}1789gitlab # [ 432.137203] gitaly[930]: time="2026-10-02T14:16:30.046Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=4 command.system_time_ms=5 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01M3YFJHG4PPBGV0XSE9K9QWCF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:30.031" grpc.time_ms=14.554 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc1790gitlab # [ 432.160247] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:30 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 901791gitlab # [ 432.254693] gitaly[930]: time="2026-10-02T14:16:30.164Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJHK1BPN44Q8N8JVEK29F grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:30.148" pid=930 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1792gitlab # [ 432.273132] gitaly[930]: time="2026-10-02T14:16:30.164Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJHK1BPN44Q8N8JVEK29F grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:30.148" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc1793gitlab # [ 432.295404] gitlab-workhorse[816]: time="2026-10-02T14:16:30Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671794gitlab # [ 432.298498] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:30 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1161795gitlab # [ 432.301305] gitaly[930]: time="2026-10-02T14:16:30.165Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJHK1BPN44Q8N8JVEK29F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:30.148" grpc.time_ms=16.251 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc1796gitlab # [ 432.465100] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:30 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1321797gitlab # [ 432.468492] gitaly[930]: time="2026-10-02T14:16:30.375Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFJHS2EWV7NRE9EFPKVTFF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:30.356" grpc.time_ms=17.403 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc1798gitlab # [ 432.581086] gitaly[930]: time="2026-10-02T14:16:30.491Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJHX9B94P5SVZVSMNKZY2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:30.472" pid=930 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc1799gitlab # [ 432.600182] gitaly[930]: time="2026-10-02T14:16:30.491Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJHX9B94P5SVZVSMNKZY2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:30.472" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc1800gitlab # [ 432.622912] gitlab-workhorse[816]: time="2026-10-02T14:16:30Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671801gitlab # [ 432.626294] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:30 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1141802gitlab # [ 432.629358] gitaly[930]: time="2026-10-02T14:16:30.491Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJHX9B94P5SVZVSMNKZY2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:30.472" grpc.time_ms=19.122 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc1803gitlab # [ 470.524091] sidekiq-cluster[3989]: 2026-10-02T14:17:08.433Z pid=3989 tid=y7x INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>3989, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>425766912}1804gitlab # [ 470.903691] sidekiq-cluster[3989]: W, [2026-10-02T14:17:08.813634 #3989] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1805gitlab # [ 471.053160] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:17:08.963Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1806gitlab # [ 474.486612] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:17:12.395Z","message":"GitLab reliable fetch activated!"}1807gitlab # [ 474.486976] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:17:12.396Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1808gitlab # [ 515.556371] postgres[5039]: [5039] ERROR: relation "table_without_models" does not exist1809gitlab # [ 515.556807] postgres[5039]: [5039] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1810gitlab # [ 520.669831] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:17:58.579Z","message":"Booted Rails 7.2.3.1 application in production environment"}1811gitlab # [ 520.673272] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:17:58.583Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1812gitlab # [ 520.676959] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:17:58.583Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1813gitlab # [ 520.679922] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:17:58.583Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1814gitlab # [ 520.683462] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:17:58.585Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1815gitlab: (finished: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails', in 102.38 seconds)1816gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1817gitlab # [ 521.331117] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:17:59 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 891818gitlab # [ 521.505468] gitaly[930]: time="2026-10-02T14:17:59.414Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=2 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFN8P9SDNM157V4EA95ZJH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=246 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:17:59.399" grpc.time_ms=15.133 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1819gitlab # Enumerating objects: 5, done.1820gitlab # [ 521.527953] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:17:59 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 201 "" "git/2.55.0" 1731821gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1822gitlab # Delta compression using up to 4 threads1823gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1824gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 273 bytes | 136.00 KiB/s, done.1825gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1826gitlab # [ 521.896223] gitaly[930]: time="2026-10-02T14:17:59.805Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=301 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=6 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFN8X2Q4B9N2AJXH1B2DWD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-10-02T14:18:29.787" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-10-02T14:17:59.788" grpc.time_ms=17.684 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1827gitlab # [ 521.922108] gitaly[930]: time="2026-10-02T14:17:59.832Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M3YFN8X2Q4B9N2AJXH1B2DWD duration_ms=153ns method=POST pid=930 status=200 url="http://unix/api/v4/internal/allowed"1828gitlab # [ 521.981494] gitaly[930]: time="2026-10-02T14:17:59.890Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M3YFN8X2Q4B9N2AJXH1B2DWD duration_ms=55ns method=POST pid=930 status=200 url="http://unix/api/v4/internal/pre_receive"1829gitlab # [ 521.995412] gitaly[930]: time="2026-10-02T14:17:59.891Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFN8X2Q4B9N2AJXH1B2DWD grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5316 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:17:59.677" grpc.time_ms=213.276 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1830gitlab # [ 522.072772] gitaly[930]: time="2026-10-02T14:17:59.982Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFN8X2Q4B9N2AJXH1B2DWD grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4786 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:17:59.981" grpc.time_ms=1.62 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1831gitlab # [ 522.180898] gitaly[930]: time="2026-10-02T14:18:00.090Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFN8X2Q4B9N2AJXH1B2DWD grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4785 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:18:00.089" grpc.time_ms=1.323 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1832gitlab # [ 522.252678] gitaly[930]: time="2026-10-02T14:18:00.162Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFN8X2Q4B9N2AJXH1B2DWD grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:18:00.161" grpc.time_ms=0.963 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1833gitlab # [ 522.530602] gitaly[930]: time="2026-10-02T14:18:00.439Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M3YFN8X2Q4B9N2AJXH1B2DWD duration_ms=207ns method=POST pid=930 status=200 url="http://unix/api/v4/internal/post_receive"1834gitlab # [ 522.540180] gitaly[930]: time="2026-10-02T14:18:00.440Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFN8X2Q4B9N2AJXH1B2DWD grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4810 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:18:00.230" grpc.time_ms=209.977 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1835gitlab # To http://gitlab/root/test-flake.git1836gitlab # [ 522.558615] gitaly[930]: time="2026-10-02T14:18:00.455Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M3YFN8X2Q4B9N2AJXH1B2DWD diskcache=ef9935ca-7efd-4a7c-9d13-c4e25f119fb7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:17:59.579" pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1837gitlab # eb0e348..727998b master -> master1838gitlab # [ 522.578137] gitlab-workhorse[816]: time="2026-10-02T14:18:00Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521839gitlab # [ 522.581748] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:18:00 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 9971840gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.41 seconds)1841gitlab: must succeed: git -C /tmp/test-flake rev-parse master1842gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.02 seconds)1843gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=727998be4a72c681de6dc36cedfc3d5e9518c280'1844gitlab # [ 522.585397] gitaly[930]: time="2026-10-02T14:18:00.455Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=423 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=851968 command.minflt=13685 command.oublock=32 command.real_time_ms=869 command.spawn_token_fork_ms=2 command.system_time_ms=278 command.user_time_ms=145 component=gitaly.StreamServerInterceptor correlation_id=01M3YFN8X2Q4B9N2AJXH1B2DWD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=605 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:17:59.579" grpc.time_ms=876.445 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1845gitlab # [ 522.703789] nixbot[4119]: INFO: ::ffff:127.0.0.1:35318 - "GET /api/repos/gitlab/root/test-flake/builds?commit=727998be4a72c681de6dc36cedfc3d5e9518c280 HTTP/1.1" 200 OK1846gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=727998be4a72c681de6dc36cedfc3d5e9518c280', in 0.10 seconds)1847gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=727998be4a72c681de6dc36cedfc3d5e9518c280'1848gitlab # [ 523.699929] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:01.608Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"a7fd28ac6841606f630f3e39","created_at":"2026-10-02T14:16:00.385Z","trace_propagation_headers":{"sentry-trace":"43e644b4f1f94479b82e338345caf292-27f50602d90c412c","baggage":"sentry-trace_id=43e644b4f1f94479b82e338345caf292,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFHKECRW6ANSJRVVH71AED","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:00.436Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-a7fd28ac6841606f630f3e39: start","job_status":"start","queue_duration_s":121.171867,"scheduling_latency_s":121.171934}1849gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=727998be4a72c681de6dc36cedfc3d5e9518c280', in 0.04 seconds)1850gitlab # [ 523.747804] nixbot[4119]: INFO: ::ffff:127.0.0.1:35326 - "GET /api/repos/gitlab/root/test-flake/builds?commit=727998be4a72c681de6dc36cedfc3d5e9518c280 HTTP/1.1" 200 OK1851gitlab # [ 523.968290] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:01.877Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"a7fd28ac6841606f630f3e39","created_at":"2026-10-02T14:16:00.385Z","trace_propagation_headers":{"sentry-trace":"43e644b4f1f94479b82e338345caf292-27f50602d90c412c","baggage":"sentry-trace_id=43e644b4f1f94479b82e338345caf292,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFHKECRW6ANSJRVVH71AED","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:00.436Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-a7fd28ac6841606f630f3e39: done: 0.268978 sec","job_status":"done","queue_duration_s":121.171867,"scheduling_latency_s":121.171934,"redis_calls":23,"redis_duration_s":0.00694,"redis_read_bytes":910,"redis_write_bytes":2540,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.003605,"redis_feature_flag_read_bytes":902,"redis_feature_flag_write_bytes":1375,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001817,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001518,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.139819,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.13258894,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.268978,"completed_at":"2026-10-02T14:18:01.877Z","load_balancing_strategy":"primary","db_duration_s":0.066013,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1852gitlab # [ 524.041365] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:01.880Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"463ed431b24c5b17ca7240fb","created_at":"2026-10-02T14:16:00.449Z","trace_propagation_headers":{"sentry-trace":"43e644b4f1f94479b82e338345caf292-27f50602d90c412c","baggage":"sentry-trace_id=43e644b4f1f94479b82e338345caf292,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFHKECRW6ANSJRVVH71AED","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:00.456Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-463ed431b24c5b17ca7240fb: start","job_status":"start","queue_duration_s":121.423308,"scheduling_latency_s":121.423343}1853gitlab # [ 524.146427] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:02.055Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"463ed431b24c5b17ca7240fb","created_at":"2026-10-02T14:16:00.449Z","trace_propagation_headers":{"sentry-trace":"43e644b4f1f94479b82e338345caf292-27f50602d90c412c","baggage":"sentry-trace_id=43e644b4f1f94479b82e338345caf292,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFHKECRW6ANSJRVVH71AED","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:00.456Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-463ed431b24c5b17ca7240fb: done: 0.176 sec","job_status":"done","queue_duration_s":121.423308,"scheduling_latency_s":121.423343,"redis_calls":14,"redis_duration_s":0.004795,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001128,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002178,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001489,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.137571,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.13192472,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.176,"completed_at":"2026-10-02T14:18:02.055Z","load_balancing_strategy":"primary","db_duration_s":0.029558,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1854gitlab # [ 524.219107] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:02.058Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"ee0dab26e6d3a73c92fc4701","created_at":"2026-10-02T14:16:01.172Z","trace_propagation_headers":{"sentry-trace":"43e644b4f1f94479b82e338345caf292-27f50602d90c412c","baggage":"sentry-trace_id=43e644b4f1f94479b82e338345caf292,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFHKECRW6ANSJRVVH71AED","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:01.181Z","job_size_bytes":5,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-ee0dab26e6d3a73c92fc4701: start","job_status":"start","queue_duration_s":120.876047,"scheduling_latency_s":120.876071}1855gitlab # [ 524.359763] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:02.269Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"ee0dab26e6d3a73c92fc4701","created_at":"2026-10-02T14:16:01.172Z","trace_propagation_headers":{"sentry-trace":"43e644b4f1f94479b82e338345caf292-27f50602d90c412c","baggage":"sentry-trace_id=43e644b4f1f94479b82e338345caf292,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFHKECRW6ANSJRVVH71AED","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:01.181Z","job_size_bytes":5,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-ee0dab26e6d3a73c92fc4701: done: 0.211463 sec","job_status":"done","queue_duration_s":120.876047,"scheduling_latency_s":120.876071,"redis_calls":12,"redis_duration_s":0.002621,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000616,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001596,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000409,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.174697,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00029836,"duration_s":0.211463,"completed_at":"2026-10-02T14:18:02.269Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.031089,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1856gitlab # [ 524.432895] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:02.271Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"c0877428894c5f8aea6e2eb8","created_at":"2026-10-02T14:16:01.221Z","trace_propagation_headers":{"sentry-trace":"43e644b4f1f94479b82e338345caf292-27f50602d90c412c","baggage":"sentry-trace_id=43e644b4f1f94479b82e338345caf292,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFHKECRW6ANSJRVVH71AED","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:01.228Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-c0877428894c5f8aea6e2eb8: start","job_status":"start","queue_duration_s":121.042469,"scheduling_latency_s":121.042488}1857gitlab # [ 524.615707] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:02.525Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"c0877428894c5f8aea6e2eb8","created_at":"2026-10-02T14:16:01.221Z","trace_propagation_headers":{"sentry-trace":"43e644b4f1f94479b82e338345caf292-27f50602d90c412c","baggage":"sentry-trace_id=43e644b4f1f94479b82e338345caf292,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFHKECRW6ANSJRVVH71AED","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:01.228Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-c0877428894c5f8aea6e2eb8: done: 0.254347 sec","job_status":"done","queue_duration_s":121.042469,"scheduling_latency_s":121.042488,"redis_calls":11,"redis_duration_s":0.002895,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000604,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001796,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000495,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.032,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.212,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.212,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.03,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.180355,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00043728,"duration_s":0.254347,"completed_at":"2026-10-02T14:18:02.525Z","load_balancing_strategy":"primary","db_duration_s":0.079229,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1858gitlab # [ 524.687058] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:02.527Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"71f9e0a3132a6595aa7d5e7b","created_at":"2026-10-02T14:16:01.232Z","trace_propagation_headers":{"sentry-trace":"43e644b4f1f94479b82e338345caf292-27f50602d90c412c","baggage":"sentry-trace_id=43e644b4f1f94479b82e338345caf292,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFHKECRW6ANSJRVVH71AED","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:01.240Z","job_size_bytes":89,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-71f9e0a3132a6595aa7d5e7b: start","job_status":"start","queue_duration_s":121.286836,"scheduling_latency_s":121.286856}1859gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=727998be4a72c681de6dc36cedfc3d5e9518c280'1860gitlab # [ 524.709914] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:02.571Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"71f9e0a3132a6595aa7d5e7b","created_at":"2026-10-02T14:16:01.232Z","trace_propagation_headers":{"sentry-trace":"43e644b4f1f94479b82e338345caf292-27f50602d90c412c","baggage":"sentry-trace_id=43e644b4f1f94479b82e338345caf292,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFHKECRW6ANSJRVVH71AED","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:01.240Z","job_size_bytes":89,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-71f9e0a3132a6595aa7d5e7b: done: 0.044731 sec","job_status":"done","queue_duration_s":121.286836,"scheduling_latency_s":121.286856,"redis_calls":11,"redis_duration_s":0.00292,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000662,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001712,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000546,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028989,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0008738,"duration_s":0.044731,"completed_at":"2026-10-02T14:18:02.571Z","load_balancing_strategy":"primary","db_duration_s":0.006754,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1861gitlab # [ 524.758730] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:02.574Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 eb0e348b96a80c3e7fb14424c8e4917e98aa1ea8 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"1249cda57798f9fa25fd0e78","created_at":"2026-10-02T14:16:12.371Z","trace_propagation_headers":{"sentry-trace":"90767057870b479387804363a8a8b6bd-01e8c5d7d94f4143","baggage":"sentry-trace_id=90767057870b479387804363a8a8b6bd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFHZP98CM7CNP4Q3XMNH64","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:12.379Z","job_size_bytes":185,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-1249cda57798f9fa25fd0e78: start","job_status":"start","queue_duration_s":110.194853,"scheduling_latency_s":110.19488}1862gitlab # [ 524.777110] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:02.598Z","message":"changes: \"0000000000000000000000000000000000000000 eb0e348b96a80c3e7fb14424c8e4917e98aa1ea8 refs/heads/master\\n\""}1863gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=727998be4a72c681de6dc36cedfc3d5e9518c280', in 0.04 seconds)1864gitlab # [ 524.780111] gitaly[930]: time="2026-10-02T14:18:02.652Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFHZP98CM7CNP4Q3XMNH64 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T14:18:12.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:18:02.639" grpc.time_ms=12.571 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1865gitlab # [ 524.794212] gitaly[930]: time="2026-10-02T14:18:02.671Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFHZP98CM7CNP4Q3XMNH64 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-02T14:18:12.028" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:18:02.659" grpc.time_ms=12.185 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1866gitlab # [ 524.809015] nixbot[4119]: INFO: ::ffff:127.0.0.1:35342 - "GET /api/repos/gitlab/root/test-flake/builds?commit=727998be4a72c681de6dc36cedfc3d5e9518c280 HTTP/1.1" 200 OK1867gitlab # [ 524.844284] gitaly[930]: time="2026-10-02T14:18:02.754Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M3YFHZP98CM7CNP4Q3XMNH64 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-10-02T14:18:12.109" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:18:02.739" grpc.time_ms=14.331 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1868gitlab # [ 524.898079] gitaly[930]: time="2026-10-02T14:18:02.808Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFHZP98CM7CNP4Q3XMNH64 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3937 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:18:02.807" grpc.time_ms=0.52 pid=930 span.kind=server system=grpc1869gitlab # [ 524.933041] gitaly[930]: time="2026-10-02T14:18:02.843Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFHZP98CM7CNP4Q3XMNH64 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3939 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:18:02.842" grpc.time_ms=0.506 pid=930 span.kind=server system=grpc1870gitlab # [ 524.949044] gitaly[930]: time="2026-10-02T14:18:02.853Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFHZP98CM7CNP4Q3XMNH64 diskcache=9e28dcb9-c0dc-4d64-af4d-b8b32bdc4e22 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-02T14:18:12.137" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:18:02.767" pid=930 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1871gitlab # [ 524.964747] gitaly[930]: time="2026-10-02T14:18:02.853Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=92 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5140 command.oublock=8 command.real_time_ms=80 command.spawn_token_fork_ms=0 command.system_time_ms=54 command.user_time_ms=38 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFHZP98CM7CNP4Q3XMNH64 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-02T14:18:12.137" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:18:02.767" grpc.time_ms=85.3 pid=930 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1872gitlab # [ 524.982270] gitaly[930]: time="2026-10-02T14:18:02.870Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFHZP98CM7CNP4Q3XMNH64 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T14:18:12.239" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:18:02.870" grpc.time_ms=0.268 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1873gitlab # [ 524.994633] gitaly[930]: time="2026-10-02T14:18:02.892Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=859 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFHZP98CM7CNP4Q3XMNH64 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-10-02T14:18:12.245" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:18:02.875" grpc.time_ms=16.88 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1874gitlab # [ 525.112838] gitaly[930]: time="2026-10-02T14:18:03.022Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=312 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFHZP98CM7CNP4Q3XMNH64 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-10-02T14:18:33.014" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:03.014" grpc.time_ms=7.867 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1875gitlab # [ 525.156697] gitaly[930]: time="2026-10-02T14:18:03.066Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFHZP98CM7CNP4Q3XMNH64 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-02T14:18:13.005" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:18:03.066" grpc.time_ms=0.442 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1876gitlab # [ 525.170181] gitaly[930]: time="2026-10-02T14:18:03.077Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=316 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01M3YFHZP98CM7CNP4Q3XMNH64 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-10-02T14:18:13.009" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-10-02T14:18:03.069" grpc.time_ms=7.803 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1877gitlab # [ 525.197620] gitaly[930]: time="2026-10-02T14:18:03.107Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFHZP98CM7CNP4Q3XMNH64 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T14:18:13.035" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:18:03.096" grpc.time_ms=11.568 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1878gitlab # [ 525.220326] gitaly[930]: time="2026-10-02T14:18:03.126Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01M3YFHZP98CM7CNP4Q3XMNH64 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T14:18:13.054" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:18:03.114" grpc.time_ms=11.498 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1879gitlab # [ 525.252182] gitaly[930]: time="2026-10-02T14:18:03.162Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=882 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=12 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M3YFHZP98CM7CNP4Q3XMNH64 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-10-02T14:18:33.142" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:03.142" grpc.time_ms=19.523 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1880gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=727998be4a72c681de6dc36cedfc3d5e9518c280'1881gitlab # [ 525.828142] nixbot[4119]: INFO: ::ffff:127.0.0.1:35346 - "GET /api/repos/gitlab/root/test-flake/builds?commit=727998be4a72c681de6dc36cedfc3d5e9518c280 HTTP/1.1" 200 OK1882gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=727998be4a72c681de6dc36cedfc3d5e9518c280', in 0.04 seconds)1883gitlab # [ 525.851683] gitaly[930]: time="2026-10-02T14:18:03.761Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFHZP98CM7CNP4Q3XMNH64 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-10-02T14:18:13.128" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:03.759" grpc.time_ms=2.045 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1884gitlab # [ 525.898955] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:03.807Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 eb0e348b96a80c3e7fb14424c8e4917e98aa1ea8 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"1249cda57798f9fa25fd0e78","created_at":"2026-10-02T14:16:12.371Z","trace_propagation_headers":{"sentry-trace":"90767057870b479387804363a8a8b6bd-01e8c5d7d94f4143","baggage":"sentry-trace_id=90767057870b479387804363a8a8b6bd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFHZP98CM7CNP4Q3XMNH64","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:12.379Z","job_size_bytes":185,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-1249cda57798f9fa25fd0e78: done: 1.233453 sec","job_status":"done","queue_duration_s":110.194853,"scheduling_latency_s":110.19488,"gitaly_calls":13,"gitaly_duration_s":0.248248,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.045864,"redis_read_bytes":2103,"redis_write_bytes":21747,"redis_cache_calls":15,"redis_cache_duration_s":0.00495,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.007015,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.001778,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7835,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.003648,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.016474,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.011999,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.069,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.296,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.333,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.069,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.882109,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00131924,"exclusive_lock_hold_duration_s":0.03851074,"duration_s":1.233453,"completed_at":"2026-10-02T14:18:03.807Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.117271,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1885gitlab # [ 525.961856] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:03.810Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"1ff664a8664f768a8370db2e","created_at":"2026-10-02T14:16:18.308Z","trace_propagation_headers":{"sentry-trace":"4e9da5c48d14450db82f1f4a123c4503-a8f2173312b64ab3","baggage":"sentry-trace_id=4e9da5c48d14450db82f1f4a123c4503,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFJ5MGM505KDHE6MBK5E16","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:18.360Z","job_size_bytes":65,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-1ff664a8664f768a8370db2e: start","job_status":"start","queue_duration_s":105.449405,"scheduling_latency_s":105.449438}1886gitlab # [ 526.013093] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:03.922Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"1ff664a8664f768a8370db2e","created_at":"2026-10-02T14:16:18.308Z","trace_propagation_headers":{"sentry-trace":"4e9da5c48d14450db82f1f4a123c4503-a8f2173312b64ab3","baggage":"sentry-trace_id=4e9da5c48d14450db82f1f4a123c4503,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFJ5MGM505KDHE6MBK5E16","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:18.360Z","job_size_bytes":65,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-1ff664a8664f768a8370db2e: done: 0.112672 sec","job_status":"done","queue_duration_s":105.449405,"scheduling_latency_s":105.449438,"redis_calls":16,"redis_duration_s":0.0043690000000000005,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.00223,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001544,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000595,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.078185,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00043418,"duration_s":0.112672,"completed_at":"2026-10-02T14:18:03.922Z","load_balancing_strategy":"primary","db_duration_s":0.025659,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1887gitlab # [ 526.068487] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:03.924Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"0ef10f0b8988e4c3e8c1ee85","created_at":"2026-10-02T14:16:18.362Z","trace_propagation_headers":{"sentry-trace":"4e9da5c48d14450db82f1f4a123c4503-a8f2173312b64ab3","baggage":"sentry-trace_id=4e9da5c48d14450db82f1f4a123c4503,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFJ5MGM505KDHE6MBK5E16","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:18.370Z","job_size_bytes":65,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-0ef10f0b8988e4c3e8c1ee85: start","job_status":"start","queue_duration_s":105.554011,"scheduling_latency_s":105.55404}1888gitlab # [ 526.151560] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:04.060Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"0ef10f0b8988e4c3e8c1ee85","created_at":"2026-10-02T14:16:18.362Z","trace_propagation_headers":{"sentry-trace":"4e9da5c48d14450db82f1f4a123c4503-a8f2173312b64ab3","baggage":"sentry-trace_id=4e9da5c48d14450db82f1f4a123c4503,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFJ5MGM505KDHE6MBK5E16","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:18.370Z","job_size_bytes":65,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-0ef10f0b8988e4c3e8c1ee85: done: 0.136245 sec","job_status":"done","queue_duration_s":105.554011,"scheduling_latency_s":105.55404,"redis_calls":21,"redis_duration_s":0.006189,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.002572,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00176,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001857,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.031,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.089273,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00058676,"duration_s":0.136245,"completed_at":"2026-10-02T14:18:04.060Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.036719,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1889gitlab # [ 526.216673] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:04.063Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"f0ed01800a3e2404164d24a1","created_at":"2026-10-02T14:16:18.521Z","trace_propagation_headers":{"sentry-trace":"4e9da5c48d14450db82f1f4a123c4503-a8f2173312b64ab3","baggage":"sentry-trace_id=4e9da5c48d14450db82f1f4a123c4503,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFJ5MGM505KDHE6MBK5E16","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:18.528Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f0ed01800a3e2404164d24a1: start","job_status":"start","queue_duration_s":105.535132,"scheduling_latency_s":105.535166}1890gitlab # [ 526.630314] gitaly[930]: time="2026-10-02T14:18:04.540Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFJ5MGM505KDHE6MBK5E16 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:18:04.539" grpc.time_ms=0.646 pid=930 span.kind=server system=grpc1891gitlab # [ 526.673926] gitaly[930]: time="2026-10-02T14:18:04.584Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFJ5MGM505KDHE6MBK5E16 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:18:04.583" grpc.time_ms=0.68 pid=930 span.kind=server system=grpc1892gitlab # [ 526.719060] gitaly[930]: time="2026-10-02T14:18:04.629Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFJ5MGM505KDHE6MBK5E16 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:18:04.628" grpc.time_ms=0.676 pid=930 span.kind=server system=grpc1893gitlab # [ 526.736694] gitaly[930]: time="2026-10-02T14:18:04.640Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ5MGM505KDHE6MBK5E16 diskcache=89daa9b0-f2da-4978-9daf-852c86d65126 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-02T14:18:34.490" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:18:04.491" pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1894gitlab # [ 526.752967] gitaly[930]: time="2026-10-02T14:18:04.641Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=162 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7589 command.oublock=0 command.real_time_ms=143 command.spawn_token_fork_ms=1 command.system_time_ms=115 command.user_time_ms=47 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ5MGM505KDHE6MBK5E16 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-02T14:18:34.490" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:18:04.491" grpc.time_ms=149.711 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1895gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=727998be4a72c681de6dc36cedfc3d5e9518c280'1896gitlab # [ 526.874015] nixbot[4119]: INFO: ::ffff:127.0.0.1:35356 - "GET /api/repos/gitlab/root/test-flake/builds?commit=727998be4a72c681de6dc36cedfc3d5e9518c280 HTTP/1.1" 200 OK1897gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=727998be4a72c681de6dc36cedfc3d5e9518c280', in 0.04 seconds)1898gitlab # [ 526.911407] gitaly[930]: time="2026-10-02T14:18:04.820Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFJ5MGM505KDHE6MBK5E16 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-02T14:18:34.818" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:04.819" grpc.time_ms=1.369 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1899gitlab # [ 526.969692] gitaly[930]: time="2026-10-02T14:18:04.879Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ5MGM505KDHE6MBK5E16 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:18:34.877" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:04.878" grpc.time_ms=1.134 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1900gitlab # [ 526.998854] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:04.906Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"f0ed01800a3e2404164d24a1","created_at":"2026-10-02T14:16:18.521Z","trace_propagation_headers":{"sentry-trace":"4e9da5c48d14450db82f1f4a123c4503-a8f2173312b64ab3","baggage":"sentry-trace_id=4e9da5c48d14450db82f1f4a123c4503,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFJ5MGM505KDHE6MBK5E16","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:18.528Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f0ed01800a3e2404164d24a1: done: 0.843342 sec","job_status":"done","queue_duration_s":105.535132,"scheduling_latency_s":105.535166,"gitaly_calls":3,"gitaly_duration_s":0.16373,"redis_calls":117,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.035182,"redis_read_bytes":455,"redis_write_bytes":27997,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.002241,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001824,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.011835,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":25,"redis_queues_duration_s":0.005077,"redis_queues_read_bytes":34,"redis_queues_write_bytes":14491,"redis_queues_metadata_calls":21,"redis_queues_metadata_duration_s":0.007486,"redis_queues_metadata_read_bytes":32,"redis_queues_metadata_write_bytes":3370,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.00118,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000679,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002619,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.078,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.035,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.041,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.04,"db_ci_duration_s":0.038,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.575157,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.7595552200000001,"duration_s":0.843342,"completed_at":"2026-10-02T14:18:04.906Z","load_balancing_strategy":"primary","db_duration_s":0.0985,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1901gitlab # [ 527.074062] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:04.909Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","eb0e348b96a80c3e7fb14424c8e4917e98aa1ea8 727998be4a72c681de6dc36cedfc3d5e9518c280 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"8a7af758acf8fc91ba2249c8","created_at":"2026-10-02T14:18:00.347Z","trace_propagation_headers":{"sentry-trace":"56a8ec8dddf24c3da2a13be0b1207792-00f701c53f494384","baggage":"sentry-trace_id=56a8ec8dddf24c3da2a13be0b1207792,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFN8X2Q4B9N2AJXH1B2DWD","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:00.366Z","job_size_bytes":185,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-8a7af758acf8fc91ba2249c8: start","job_status":"start","queue_duration_s":4.543057,"scheduling_latency_s":4.543093}1902gitlab # [ 527.092896] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:04.925Z","message":"changes: \"eb0e348b96a80c3e7fb14424c8e4917e98aa1ea8 727998be4a72c681de6dc36cedfc3d5e9518c280 refs/heads/master\\n\""}1903gitlab # [ 527.095676] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:04.981Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"491a67319a11b51347952338","created_at":"2026-10-02T14:18:04.978Z","trace_propagation_headers":{"sentry-trace":"56a8ec8dddf24c3da2a13be0b1207792-1a0151fe36c249b4","baggage":"sentry-trace_id=56a8ec8dddf24c3da2a13be0b1207792,sentry-release=Unknown"},"correlation_id":"01M3YFN8X2Q4B9N2AJXH1B2DWD","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"b389ffd83e3a15368f95332e","job_size_bytes":46,"pid":3989,"job_status":"deduplicated","message":"ProjectCacheWorker JID-491a67319a11b51347952338: deduplicated: until executing","deduplication.type":"until executing"}1904gitlab # [ 527.112439] gitaly[930]: time="2026-10-02T14:18:04.959Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFN8X2Q4B9N2AJXH1B2DWD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T14:18:14.007" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:18:04.947" grpc.time_ms=12.182 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1905gitlab # [ 527.126716] gitaly[930]: time="2026-10-02T14:18:04.976Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFN8X2Q4B9N2AJXH1B2DWD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-02T14:18:14.025" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:18:04.965" grpc.time_ms=10.314 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1906gitlab # [ 527.141226] gitaly[930]: time="2026-10-02T14:18:05.007Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFN8X2Q4B9N2AJXH1B2DWD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-10-02T14:18:14.056" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:18:04.996" grpc.time_ms=10.603 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1907gitlab # [ 527.155639] gitaly[930]: time="2026-10-02T14:18:05.024Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=597 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFN8X2Q4B9N2AJXH1B2DWD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-10-02T14:18:35.012" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:05.012" grpc.time_ms=11.71 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1908gitlab # [ 527.172270] gitaly[930]: time="2026-10-02T14:18:05.045Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=318 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01M3YFN8X2Q4B9N2AJXH1B2DWD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-10-02T14:18:15.008" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-10-02T14:18:05.038" grpc.time_ms=6.717 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1909gitlab # [ 527.186490] gitaly[930]: time="2026-10-02T14:18:05.067Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFN8X2Q4B9N2AJXH1B2DWD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T14:18:15.026" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:18:05.056" grpc.time_ms=10.239 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1910gitlab # [ 527.280410] gitaly[930]: time="2026-10-02T14:18:05.190Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFN8X2Q4B9N2AJXH1B2DWD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-10-02T14:18:15.128" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:05.188" grpc.time_ms=1.378 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1911gitlab # [ 527.308987] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:05.218Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","eb0e348b96a80c3e7fb14424c8e4917e98aa1ea8 727998be4a72c681de6dc36cedfc3d5e9518c280 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"8a7af758acf8fc91ba2249c8","created_at":"2026-10-02T14:18:00.347Z","trace_propagation_headers":{"sentry-trace":"56a8ec8dddf24c3da2a13be0b1207792-00f701c53f494384","baggage":"sentry-trace_id=56a8ec8dddf24c3da2a13be0b1207792,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFN8X2Q4B9N2AJXH1B2DWD","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:00.366Z","job_size_bytes":185,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-8a7af758acf8fc91ba2249c8: done: 0.308771 sec","job_status":"done","queue_duration_s":4.543057,"scheduling_latency_s":4.543093,"gitaly_calls":7,"gitaly_duration_s":0.083815,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.021594,"redis_read_bytes":501,"redis_write_bytes":11941,"redis_cache_calls":9,"redis_cache_duration_s":0.002383,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.001058,"redis_queues_read_bytes":9,"redis_queues_write_bytes":5718,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.003143,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.006808,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.008202,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.03,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.03,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.03,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.19291,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00081872,"exclusive_lock_hold_duration_s":0.02319778,"duration_s":0.308771,"completed_at":"2026-10-02T14:18:05.218Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.031596,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1912gitlab # [ 527.377081] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:05.220Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"b389ffd83e3a15368f95332e","created_at":"2026-10-02T14:18:02.675Z","trace_propagation_headers":{"sentry-trace":"90767057870b479387804363a8a8b6bd-ab5b24c38e8f47d6","baggage":"sentry-trace_id=90767057870b479387804363a8a8b6bd,sentry-release=Unknown"},"correlation_id":"01M3YFHZP98CM7CNP4Q3XMNH64","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:02.705Z","job_size_bytes":46,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-b389ffd83e3a15368f95332e: start","job_status":"start","queue_duration_s":2.514479,"scheduling_latency_s":2.51451}1913gitlab # [ 527.394535] gitaly[930]: time="2026-10-02T14:18:05.285Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFHZP98CM7CNP4Q3XMNH64 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-10-02T20:18:05.278" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:18:05.278" grpc.time_ms=6.238 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1914gitlab # [ 527.406182] gitaly[930]: time="2026-10-02T14:18:05.308Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=596 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFHZP98CM7CNP4Q3XMNH64 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-10-02T14:18:35.296" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:05.297" grpc.time_ms=10.821 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1915gitlab # [ 527.437096] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:05.346Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"b389ffd83e3a15368f95332e","created_at":"2026-10-02T14:18:02.675Z","trace_propagation_headers":{"sentry-trace":"90767057870b479387804363a8a8b6bd-ab5b24c38e8f47d6","baggage":"sentry-trace_id=90767057870b479387804363a8a8b6bd,sentry-release=Unknown"},"correlation_id":"01M3YFHZP98CM7CNP4Q3XMNH64","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:02.705Z","job_size_bytes":46,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-b389ffd83e3a15368f95332e: done: 0.12617 sec","job_status":"done","queue_duration_s":2.514479,"scheduling_latency_s":2.51451,"gitaly_calls":2,"gitaly_duration_s":0.022997,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.007647,"redis_read_bytes":69,"redis_write_bytes":4886,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001345,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.00063,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2114,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001695,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.003219,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":753,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000758,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.082099,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00036972,"duration_s":0.12617,"completed_at":"2026-10-02T14:18:05.346Z","load_balancing_strategy":"primary","db_duration_s":0.014018,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1916gitlab # [ 527.496868] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:05.348Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"cb7b72d796f050803acaa952","created_at":"2026-10-02T14:18:02.728Z","trace_propagation_headers":{"sentry-trace":"90767057870b479387804363a8a8b6bd-ab5b24c38e8f47d6","baggage":"sentry-trace_id=90767057870b479387804363a8a8b6bd,sentry-release=Unknown"},"correlation_id":"01M3YFHZP98CM7CNP4Q3XMNH64","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:02.735Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-cb7b72d796f050803acaa952: start","job_status":"start","queue_duration_s":2.613465,"scheduling_latency_s":2.613494}1917gitlab # [ 527.515123] gitaly[930]: time="2026-10-02T14:18:05.405Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFHZP98CM7CNP4Q3XMNH64 diskcache=f96dffea-abac-4fcd-aff5-5a120c118dc0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-02T20:18:05.379" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:05.379" pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1918gitlab # [ 527.531991] gitaly[930]: time="2026-10-02T14:18:05.406Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1175 command.oublock=0 command.real_time_ms=29 command.spawn_token_fork_ms=0 command.system_time_ms=15 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFHZP98CM7CNP4Q3XMNH64 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-02T20:18:05.379" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:05.379" grpc.time_ms=25.937 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1919gitlab # [ 527.557263] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:05.454Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"cb7b72d796f050803acaa952","created_at":"2026-10-02T14:18:02.728Z","trace_propagation_headers":{"sentry-trace":"90767057870b479387804363a8a8b6bd-ab5b24c38e8f47d6","baggage":"sentry-trace_id=90767057870b479387804363a8a8b6bd,sentry-release=Unknown"},"correlation_id":"01M3YFHZP98CM7CNP4Q3XMNH64","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:02.735Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-cb7b72d796f050803acaa952: done: 0.10633 sec","job_status":"done","queue_duration_s":2.613465,"scheduling_latency_s":2.613494,"gitaly_calls":1,"gitaly_duration_s":0.029045,"redis_calls":15,"redis_duration_s":0.0035,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00043,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001243,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000647,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00118,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.013,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.025,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.052815,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0753129,"duration_s":0.10633,"completed_at":"2026-10-02T14:18:05.454Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.018855,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1920gitlab # [ 527.617870] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:05.456Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"d9a4a6cd8663bffaf302d1db","created_at":"2026-10-02T14:18:02.864Z","trace_propagation_headers":{"sentry-trace":"90767057870b479387804363a8a8b6bd-ab5b24c38e8f47d6","baggage":"sentry-trace_id=90767057870b479387804363a8a8b6bd,sentry-release=Unknown"},"correlation_id":"01M3YFHZP98CM7CNP4Q3XMNH64","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:02.867Z","job_size_bytes":23,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-d9a4a6cd8663bffaf302d1db: start","job_status":"start","queue_duration_s":2.58953,"scheduling_latency_s":2.589559}1921gitlab # [ 527.634118] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:05.539Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"d9a4a6cd8663bffaf302d1db","created_at":"2026-10-02T14:18:02.864Z","trace_propagation_headers":{"sentry-trace":"90767057870b479387804363a8a8b6bd-ab5b24c38e8f47d6","baggage":"sentry-trace_id=90767057870b479387804363a8a8b6bd,sentry-release=Unknown"},"correlation_id":"01M3YFHZP98CM7CNP4Q3XMNH64","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:02.867Z","job_size_bytes":23,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-d9a4a6cd8663bffaf302d1db: done: 0.082521 sec","job_status":"done","queue_duration_s":2.58953,"scheduling_latency_s":2.589559,"gitaly_calls":1,"gitaly_duration_s":0.013048,"redis_calls":16,"redis_duration_s":0.003985,"redis_read_bytes":52,"redis_write_bytes":2047,"redis_queues_calls":1,"redis_queues_duration_s":0.000363,"redis_queues_read_bytes":1,"redis_queues_write_bytes":978,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001176,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.001689,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000757,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.05028,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.082521,"completed_at":"2026-10-02T14:18:05.539Z","load_balancing_strategy":"primary","db_duration_s":0.01502,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1922gitlab # [ 527.681276] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:05.541Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"53aa4b9549e956e6dd6118f1","created_at":"2026-10-02T14:18:03.529Z","trace_propagation_headers":{"sentry-trace":"90767057870b479387804363a8a8b6bd-ab5b24c38e8f47d6","baggage":"sentry-trace_id=90767057870b479387804363a8a8b6bd,sentry-release=Unknown"},"correlation_id":"01M3YFHZP98CM7CNP4Q3XMNH64","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b9fa564f752e06c1cd3094931e5269c83d1b771d33cef4610bd36354d9510a07","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:03.538Z","job_size_bytes":467,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-53aa4b9549e956e6dd6118f1: start","job_status":"start","queue_duration_s":2.002509,"scheduling_latency_s":2.002546}1923gitlab # [ 527.700280] gitaly[930]: time="2026-10-02T14:18:05.524Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=594 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFHZP98CM7CNP4Q3XMNH64 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-10-02T14:18:35.513" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:05.514" grpc.time_ms=9.902 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1924gitlab # [ 527.713936] gitaly[930]: time="2026-10-02T14:18:05.581Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFHZP98CM7CNP4Q3XMNH64 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:18:35.579" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:05.580" grpc.time_ms=0.774 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1925gitlab # [ 527.726999] gitaly[930]: time="2026-10-02T14:18:05.595Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFHZP98CM7CNP4Q3XMNH64 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-02T14:18:15.004" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:18:05.584" grpc.time_ms=10.241 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1926gitlab # [ 527.838954] gitaly[930]: time="2026-10-02T14:18:05.749Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFHZP98CM7CNP4Q3XMNH64 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-02T14:18:35.047" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:05.747" grpc.time_ms=0.995 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1927gitlab # [ 527.852706] gitaly[930]: time="2026-10-02T14:18:05.757Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=34 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M3YFHZP98CM7CNP4Q3XMNH64 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-10-02T14:18:35.054" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:05.754" grpc.time_ms=3.087 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1928gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=727998be4a72c681de6dc36cedfc3d5e9518c280'1929gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=727998be4a72c681de6dc36cedfc3d5e9518c280', in 0.04 seconds)1930gitlab # [ 527.913207] nixbot[4119]: INFO: ::ffff:127.0.0.1:35364 - "GET /api/repos/gitlab/root/test-flake/builds?commit=727998be4a72c681de6dc36cedfc3d5e9518c280 HTTP/1.1" 200 OK1931gitlab # [ 527.984500] gitaly[930]: time="2026-10-02T14:18:05.893Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFHZP98CM7CNP4Q3XMNH64 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T14:18:15.001" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:18:05.881" grpc.time_ms=11.952 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1932gitlab # [ 528.859862] gitaly[930]: time="2026-10-02T14:18:06.769Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=880 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01M3YFHZP98CM7CNP4Q3XMNH64 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-10-02T14:18:36.048" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:06.749" grpc.time_ms=20.351 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1933gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=727998be4a72c681de6dc36cedfc3d5e9518c280'1934gitlab # [ 528.880350] gitaly[930]: time="2026-10-02T14:18:06.790Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M3YFHZP98CM7CNP4Q3XMNH64 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-10-02T14:18:36.074" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:06.774" grpc.time_ms=15.45 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1935gitlab # [ 528.916679] sidekiq-cluster[3989]: {"severity":"WARN","time":"2026-10-02T14:18:06.825Z","class":"CreatePipelineWorker","correlation_id":"01M3YFHZP98CM7CNP4Q3XMNH64","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"eb0e348b96a80c3e7fb14424c8e4917e98aa1ea8","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"eb0e348b96a80c3e7fb14424c8e4917e98aa1ea8","ref":"refs/heads/master"},"retry":0}1936gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=727998be4a72c681de6dc36cedfc3d5e9518c280', in 0.10 seconds)1937gitlab # [ 528.952298] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:06.834Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"53aa4b9549e956e6dd6118f1","created_at":"2026-10-02T14:18:03.529Z","trace_propagation_headers":{"sentry-trace":"90767057870b479387804363a8a8b6bd-ab5b24c38e8f47d6","baggage":"sentry-trace_id=90767057870b479387804363a8a8b6bd,sentry-release=Unknown"},"correlation_id":"01M3YFHZP98CM7CNP4Q3XMNH64","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b9fa564f752e06c1cd3094931e5269c83d1b771d33cef4610bd36354d9510a07","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:03.538Z","job_size_bytes":467,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-53aa4b9549e956e6dd6118f1: done: 1.29274 sec","job_status":"done","queue_duration_s":2.002509,"scheduling_latency_s":2.002546,"gitaly_calls":7,"gitaly_duration_s":0.084604,"redis_calls":51,"redis_duration_s":0.012346000000000001,"redis_read_bytes":169,"redis_write_bytes":4168,"redis_cache_calls":4,"redis_cache_duration_s":0.000959,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.005647,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001703,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002958,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001079,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.043,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.037,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":1.142156,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0005897,"duration_s":1.29274,"completed_at":"2026-10-02T14:18:06.834Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.072517,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1938gitlab # [ 529.020542] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:06.836Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"f0edb3ef-1c89-482c-9b90-e6f6a247da8f\"}"],"class":"WebHookWorker","jid":"2143c77444fa7f4a0b86f27a","created_at":"2026-10-02T14:18:03.718Z","trace_propagation_headers":{"sentry-trace":"90767057870b479387804363a8a8b6bd-ab5b24c38e8f47d6","baggage":"sentry-trace_id=90767057870b479387804363a8a8b6bd,sentry-release=Unknown"},"correlation_id":"01M3YFHZP98CM7CNP4Q3XMNH64","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:df20922c4826b9e728c54571b190db01a41bb69f600365ef5373fc61938c393c","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:03.725Z","job_size_bytes":1751,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-2143c77444fa7f4a0b86f27a: start","job_status":"start","queue_duration_s":3.110506,"scheduling_latency_s":3.110531}1939gitlab # [ 529.041632] nixbot[4119]: INFO: ::ffff:127.0.0.1:35378 - "GET /api/repos/gitlab/root/test-flake/builds?commit=727998be4a72c681de6dc36cedfc3d5e9518c280 HTTP/1.1" 200 OK1940gitlab # [ 529.115543] nixbot[4119]: INFO: ::ffff:127.0.0.1:35390 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted1941gitlab # [ 529.141828] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:07.050Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"f0edb3ef-1c89-482c-9b90-e6f6a247da8f\"}"],"class":"WebHookWorker","jid":"2143c77444fa7f4a0b86f27a","created_at":"2026-10-02T14:18:03.718Z","trace_propagation_headers":{"sentry-trace":"90767057870b479387804363a8a8b6bd-ab5b24c38e8f47d6","baggage":"sentry-trace_id=90767057870b479387804363a8a8b6bd,sentry-release=Unknown"},"correlation_id":"01M3YFHZP98CM7CNP4Q3XMNH64","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:df20922c4826b9e728c54571b190db01a41bb69f600365ef5373fc61938c393c","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:03.725Z","job_size_bytes":1751,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-2143c77444fa7f4a0b86f27a: done: 0.21422 sec","job_status":"done","queue_duration_s":3.110506,"scheduling_latency_s":3.110531,"redis_calls":19,"redis_duration_s":0.004837,"redis_read_bytes":32,"redis_write_bytes":5368,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001236,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000581,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3730,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001919,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001101,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.07121329999995396,"cpu_s":0.112821,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003975,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.21422,"completed_at":"2026-10-02T14:18:07.050Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00714,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1942gitlab # [ 529.216054] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:07.053Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"afceaa2746469297dfad2e54","created_at":"2026-10-02T14:18:04.459Z","trace_propagation_headers":{"sentry-trace":"4e9da5c48d14450db82f1f4a123c4503-75ceaa868373473c","baggage":"sentry-trace_id=4e9da5c48d14450db82f1f4a123c4503,sentry-release=Unknown"},"correlation_id":"01M3YFJ5MGM505KDHE6MBK5E16","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:04.468Z","job_size_bytes":23,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-afceaa2746469297dfad2e54: start","job_status":"start","queue_duration_s":2.58465,"scheduling_latency_s":2.584682}1943gitlab # [ 529.249194] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:18:07 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 911944gitlab # [ 529.252245] gitaly[930]: time="2026-10-02T14:18:07.158Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFNGATNKGKW5NW585TPA8T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:18:07.145" grpc.time_ms=12.073 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc1945gitlab # [ 529.275741] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:07.185Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"afceaa2746469297dfad2e54","created_at":"2026-10-02T14:18:04.459Z","trace_propagation_headers":{"sentry-trace":"4e9da5c48d14450db82f1f4a123c4503-75ceaa868373473c","baggage":"sentry-trace_id=4e9da5c48d14450db82f1f4a123c4503,sentry-release=Unknown"},"correlation_id":"01M3YFJ5MGM505KDHE6MBK5E16","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:04.468Z","job_size_bytes":23,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-afceaa2746469297dfad2e54: done: 0.131607 sec","job_status":"done","queue_duration_s":2.58465,"scheduling_latency_s":2.584682,"redis_calls":10,"redis_duration_s":0.002958,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000556,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00176,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000642,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.008,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.008,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.100782,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00046568,"duration_s":0.131607,"completed_at":"2026-10-02T14:18:07.185Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.023722,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1946gitlab # [ 529.325689] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:07.187Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"370a1a6049fbf318ff7a1d37","created_at":"2026-10-02T14:18:04.650Z","trace_propagation_headers":{"sentry-trace":"4e9da5c48d14450db82f1f4a123c4503-75ceaa868373473c","baggage":"sentry-trace_id=4e9da5c48d14450db82f1f4a123c4503,sentry-release=Unknown"},"correlation_id":"01M3YFJ5MGM505KDHE6MBK5E16","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:04.659Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-370a1a6049fbf318ff7a1d37: start","job_status":"start","queue_duration_s":2.528554,"scheduling_latency_s":2.528579}1947gitlab # [ 529.344611] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:07.238Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"370a1a6049fbf318ff7a1d37","created_at":"2026-10-02T14:18:04.650Z","trace_propagation_headers":{"sentry-trace":"4e9da5c48d14450db82f1f4a123c4503-75ceaa868373473c","baggage":"sentry-trace_id=4e9da5c48d14450db82f1f4a123c4503,sentry-release=Unknown"},"correlation_id":"01M3YFJ5MGM505KDHE6MBK5E16","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:04.659Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-370a1a6049fbf318ff7a1d37: done: 0.050883 sec","job_status":"done","queue_duration_s":2.528554,"scheduling_latency_s":2.528579,"redis_calls":10,"redis_duration_s":0.002279,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000491,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001324,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000464,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034482,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00036978,"duration_s":0.050883,"completed_at":"2026-10-02T14:18:07.238Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007776,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1948gitlab # [ 529.392412] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:07.240Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"41d3ab2ac03d83464f7b2295","created_at":"2026-10-02T14:18:04.679Z","trace_propagation_headers":{"sentry-trace":"4e9da5c48d14450db82f1f4a123c4503-75ceaa868373473c","baggage":"sentry-trace_id=4e9da5c48d14450db82f1f4a123c4503,sentry-release=Unknown"},"correlation_id":"01M3YFJ5MGM505KDHE6MBK5E16","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:04.688Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-41d3ab2ac03d83464f7b2295: start","job_status":"start","queue_duration_s":2.552273,"scheduling_latency_s":2.552297}1949gitlab # [ 529.409497] gitaly[930]: time="2026-10-02T14:18:07.253Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNGDRZ0Q9CBDMF663VHW2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:18:07.239" pid=930 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1950gitlab # [ 529.420501] gitlab-workhorse[816]: time="2026-10-02T14:18:07Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671951gitlab # [ 529.422493] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:18:07 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 931952gitlab # [ 529.424636] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:07.297Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"41d3ab2ac03d83464f7b2295","created_at":"2026-10-02T14:18:04.679Z","trace_propagation_headers":{"sentry-trace":"4e9da5c48d14450db82f1f4a123c4503-75ceaa868373473c","baggage":"sentry-trace_id=4e9da5c48d14450db82f1f4a123c4503,sentry-release=Unknown"},"correlation_id":"01M3YFJ5MGM505KDHE6MBK5E16","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:04.688Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-41d3ab2ac03d83464f7b2295: done: 0.056183 sec","job_status":"done","queue_duration_s":2.552273,"scheduling_latency_s":2.552297,"redis_calls":10,"redis_duration_s":0.004661,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001996,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001978,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000687,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036435,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00042632,"duration_s":0.056183,"completed_at":"2026-10-02T14:18:07.296Z","load_balancing_strategy":"primary","db_duration_s":0.008687,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1953gitlab # [ 529.472145] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:07.299Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"e96207385e8d16adab75ade9","created_at":"2026-10-02T14:18:04.708Z","trace_propagation_headers":{"sentry-trace":"4e9da5c48d14450db82f1f4a123c4503-75ceaa868373473c","baggage":"sentry-trace_id=4e9da5c48d14450db82f1f4a123c4503,sentry-release=Unknown"},"correlation_id":"01M3YFJ5MGM505KDHE6MBK5E16","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:04.715Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-e96207385e8d16adab75ade9: start","job_status":"start","queue_duration_s":2.5836,"scheduling_latency_s":2.583696}1954gitlab # [ 529.489054] gitaly[930]: time="2026-10-02T14:18:07.253Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNGDRZ0Q9CBDMF663VHW2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:18:07.239" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc1955gitlab # [ 529.503631] gitlab-workhorse[816]: time="2026-10-02T14:18:07Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3371956gitlab # [ 529.505773] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:18:07 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 337 "" "git/2.55.0" 981957gitlab # [ 529.507846] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:07.363Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"e96207385e8d16adab75ade9","created_at":"2026-10-02T14:18:04.708Z","trace_propagation_headers":{"sentry-trace":"4e9da5c48d14450db82f1f4a123c4503-75ceaa868373473c","baggage":"sentry-trace_id=4e9da5c48d14450db82f1f4a123c4503,sentry-release=Unknown"},"correlation_id":"01M3YFJ5MGM505KDHE6MBK5E16","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:04.715Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-e96207385e8d16adab75ade9: done: 0.064928 sec","job_status":"done","queue_duration_s":2.5836,"scheduling_latency_s":2.583696,"redis_calls":13,"redis_duration_s":0.0036569999999999997,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000547,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001516,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000949,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000645,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.044034,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00043466,"duration_s":0.064928,"completed_at":"2026-10-02T14:18:07.363Z","load_balancing_strategy":"primary","db_duration_s":0.009669,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1958gitlab # [ 529.557473] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:07.366Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"6c07abf9045131e22576fa42","created_at":"2026-10-02T14:18:04.716Z","trace_propagation_headers":{"sentry-trace":"4e9da5c48d14450db82f1f4a123c4503-75ceaa868373473c","baggage":"sentry-trace_id=4e9da5c48d14450db82f1f4a123c4503,sentry-release=Unknown"},"correlation_id":"01M3YFJ5MGM505KDHE6MBK5E16","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:04.722Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-6c07abf9045131e22576fa42: start","job_status":"start","queue_duration_s":2.643939,"scheduling_latency_s":2.643965}1959gitlab # [ 529.575394] gitaly[930]: time="2026-10-02T14:18:07.253Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNGDRZ0Q9CBDMF663VHW2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:18:07.239" grpc.time_ms=14.021 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc1960gitlab # [ 529.591215] nixbot[4119]: {"ts": "2026-10-02T14:18:07.403Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"}1961gitlab # [ 529.594007] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:07.428Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"6c07abf9045131e22576fa42","created_at":"2026-10-02T14:18:04.716Z","trace_propagation_headers":{"sentry-trace":"4e9da5c48d14450db82f1f4a123c4503-75ceaa868373473c","baggage":"sentry-trace_id=4e9da5c48d14450db82f1f4a123c4503,sentry-release=Unknown"},"correlation_id":"01M3YFJ5MGM505KDHE6MBK5E16","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:04.722Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-6c07abf9045131e22576fa42: done: 0.062733 sec","job_status":"done","queue_duration_s":2.643939,"scheduling_latency_s":2.643965,"redis_calls":10,"redis_duration_s":0.004523,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002428,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001504,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000591,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039266,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00033796,"duration_s":0.062733,"completed_at":"2026-10-02T14:18:07.428Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008905,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1962gitlab # [ 529.641969] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:07.431Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"85a239dcfcf2b427e61247a1","created_at":"2026-10-02T14:18:04.722Z","trace_propagation_headers":{"sentry-trace":"4e9da5c48d14450db82f1f4a123c4503-75ceaa868373473c","baggage":"sentry-trace_id=4e9da5c48d14450db82f1f4a123c4503,sentry-release=Unknown"},"correlation_id":"01M3YFJ5MGM505KDHE6MBK5E16","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:04.730Z","job_size_bytes":19,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-85a239dcfcf2b427e61247a1: start","job_status":"start","queue_duration_s":2.7011,"scheduling_latency_s":2.701131}1963gitlab # [ 529.659479] gitaly[930]: time="2026-10-02T14:18:07.354Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNGGSE2ZWVJ79W84CYP2P grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:18:07.331" pid=930 remote_ip="2001:db8:1::1" request_sha=30e0643f9fa22223c5f3693ab3272c7091675047 response_bytes=337 span.kind=server system=grpc1964gitlab # [ 529.670670] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:07.482Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"85a239dcfcf2b427e61247a1","created_at":"2026-10-02T14:18:04.722Z","trace_propagation_headers":{"sentry-trace":"4e9da5c48d14450db82f1f4a123c4503-75ceaa868373473c","baggage":"sentry-trace_id=4e9da5c48d14450db82f1f4a123c4503,sentry-release=Unknown"},"correlation_id":"01M3YFJ5MGM505KDHE6MBK5E16","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:04.730Z","job_size_bytes":19,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-85a239dcfcf2b427e61247a1: done: 0.051204 sec","job_status":"done","queue_duration_s":2.7011,"scheduling_latency_s":2.701131,"redis_calls":11,"redis_duration_s":0.003065,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000589,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001972,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000504,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034353,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003778,"duration_s":0.051204,"completed_at":"2026-10-02T14:18:07.482Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006065,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1965gitlab # [ 529.718641] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:07.484Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"eec2e7637871ba9430be1f5f","created_at":"2026-10-02T14:18:04.733Z","trace_propagation_headers":{"sentry-trace":"4e9da5c48d14450db82f1f4a123c4503-75ceaa868373473c","baggage":"sentry-trace_id=4e9da5c48d14450db82f1f4a123c4503,sentry-release=Unknown"},"correlation_id":"01M3YFJ5MGM505KDHE6MBK5E16","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:04.739Z","job_size_bytes":66,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-eec2e7637871ba9430be1f5f: start","job_status":"start","queue_duration_s":2.744922,"scheduling_latency_s":2.744949}1966gitlab # [ 529.737592] gitaly[930]: time="2026-10-02T14:18:07.355Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1129 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=4 command.system_time_ms=12 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNGGSE2ZWVJ79W84CYP2P grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:18:07.331" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc1967gitlab # [ 529.753442] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:07.531Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"eec2e7637871ba9430be1f5f","created_at":"2026-10-02T14:18:04.733Z","trace_propagation_headers":{"sentry-trace":"4e9da5c48d14450db82f1f4a123c4503-75ceaa868373473c","baggage":"sentry-trace_id=4e9da5c48d14450db82f1f4a123c4503,sentry-release=Unknown"},"correlation_id":"01M3YFJ5MGM505KDHE6MBK5E16","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:04.739Z","job_size_bytes":66,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-eec2e7637871ba9430be1f5f: done: 0.047141 sec","job_status":"done","queue_duration_s":2.744922,"scheduling_latency_s":2.744949,"redis_calls":10,"redis_duration_s":0.002326,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000476,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001344,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000506,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030614,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00036328,"duration_s":0.047141,"completed_at":"2026-10-02T14:18:07.531Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006979,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1968gitlab # [ 529.802058] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:07.533Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"b117134beb347fd474dab440","created_at":"2026-10-02T14:18:04.740Z","trace_propagation_headers":{"sentry-trace":"4e9da5c48d14450db82f1f4a123c4503-75ceaa868373473c","baggage":"sentry-trace_id=4e9da5c48d14450db82f1f4a123c4503,sentry-release=Unknown"},"correlation_id":"01M3YFJ5MGM505KDHE6MBK5E16","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:04.747Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-b117134beb347fd474dab440: start","job_status":"start","queue_duration_s":2.785954,"scheduling_latency_s":2.785978}1969gitlab # [ 529.819513] gitaly[930]: time="2026-10-02T14:18:07.355Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1129 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=4 command.system_time_ms=12 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNGGSE2ZWVJ79W84CYP2P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:18:07.331" grpc.time_ms=23.862 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc1970gitlab # [ 529.835724] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:07.591Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"b117134beb347fd474dab440","created_at":"2026-10-02T14:18:04.740Z","trace_propagation_headers":{"sentry-trace":"4e9da5c48d14450db82f1f4a123c4503-75ceaa868373473c","baggage":"sentry-trace_id=4e9da5c48d14450db82f1f4a123c4503,sentry-release=Unknown"},"correlation_id":"01M3YFJ5MGM505KDHE6MBK5E16","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:04.747Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-b117134beb347fd474dab440: done: 0.058336 sec","job_status":"done","queue_duration_s":2.785954,"scheduling_latency_s":2.785978,"redis_calls":10,"redis_duration_s":0.002312,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000502,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001346,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000464,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038268,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00030944,"duration_s":0.058336,"completed_at":"2026-10-02T14:18:07.591Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010761,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1971gitlab # [ 529.883541] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:07.593Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"5b753a819f674f7078488372","created_at":"2026-10-02T14:18:04.756Z","trace_propagation_headers":{"sentry-trace":"4e9da5c48d14450db82f1f4a123c4503-75ceaa868373473c","baggage":"sentry-trace_id=4e9da5c48d14450db82f1f4a123c4503,sentry-release=Unknown"},"correlation_id":"01M3YFJ5MGM505KDHE6MBK5E16","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:04.764Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-5b753a819f674f7078488372: start","job_status":"start","queue_duration_s":2.828697,"scheduling_latency_s":2.828722}1972gitlab # [ 529.901406] gitaly[930]: time="2026-10-02T14:18:07.709Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNGNVJJXYADCQMCTD2X99 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:18:37.708" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:07.708" grpc.time_ms=0.899 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1973gitlab # [ 529.915087] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:07.637Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"5b753a819f674f7078488372","created_at":"2026-10-02T14:18:04.756Z","trace_propagation_headers":{"sentry-trace":"4e9da5c48d14450db82f1f4a123c4503-75ceaa868373473c","baggage":"sentry-trace_id=4e9da5c48d14450db82f1f4a123c4503,sentry-release=Unknown"},"correlation_id":"01M3YFJ5MGM505KDHE6MBK5E16","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:04.764Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-5b753a819f674f7078488372: done: 0.044618 sec","job_status":"done","queue_duration_s":2.828697,"scheduling_latency_s":2.828722,"redis_calls":10,"redis_duration_s":0.002546,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000697,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001263,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000586,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027297,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00039074,"duration_s":0.044618,"completed_at":"2026-10-02T14:18:07.637Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010584,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1974gitlab # [ 529.964599] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:07.639Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"85419ffdf4f34f5d345c88bd","created_at":"2026-10-02T14:18:04.765Z","trace_propagation_headers":{"sentry-trace":"4e9da5c48d14450db82f1f4a123c4503-75ceaa868373473c","baggage":"sentry-trace_id=4e9da5c48d14450db82f1f4a123c4503,sentry-release=Unknown"},"correlation_id":"01M3YFJ5MGM505KDHE6MBK5E16","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:04.773Z","job_size_bytes":6,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-85419ffdf4f34f5d345c88bd: start","job_status":"start","queue_duration_s":2.865643,"scheduling_latency_s":2.865667}1975gitlab # [ 529.981766] gitaly[930]: time="2026-10-02T14:18:07.802Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFNGNVJJXYADCQMCTD2X99 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T14:18:17.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:18:07.790" grpc.time_ms=12.225 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1976gitlab # [ 529.996334] nixbot[4119]: {"ts": "2026-10-02T14:18:07.881Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/eb0e348b96a80c3e7fb14424c8e4917e98aa1ea8 \"HTTP/1.1 201 Created\""}1977gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=727998be4a72c681de6dc36cedfc3d5e9518c280'1978gitlab # [ 529.999360] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:07.668Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"85419ffdf4f34f5d345c88bd","created_at":"2026-10-02T14:18:04.765Z","trace_propagation_headers":{"sentry-trace":"4e9da5c48d14450db82f1f4a123c4503-75ceaa868373473c","baggage":"sentry-trace_id=4e9da5c48d14450db82f1f4a123c4503,sentry-release=Unknown"},"correlation_id":"01M3YFJ5MGM505KDHE6MBK5E16","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:04.773Z","job_size_bytes":6,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-85419ffdf4f34f5d345c88bd: done: 0.028864 sec","job_status":"done","queue_duration_s":2.865643,"scheduling_latency_s":2.865667,"redis_calls":10,"redis_duration_s":0.002841,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00047,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00133,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001041,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016935,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00035766,"duration_s":0.028864,"completed_at":"2026-10-02T14:18:07.668Z","load_balancing_strategy":"primary","db_duration_s":0.003106,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1979gitlab # [ 530.045956] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:07.670Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"7b1add2e4d1e1d7df8791624","created_at":"2026-10-02T14:18:04.774Z","trace_propagation_headers":{"sentry-trace":"4e9da5c48d14450db82f1f4a123c4503-75ceaa868373473c","baggage":"sentry-trace_id=4e9da5c48d14450db82f1f4a123c4503,sentry-release=Unknown"},"correlation_id":"01M3YFJ5MGM505KDHE6MBK5E16","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:04.782Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-7b1add2e4d1e1d7df8791624: start","job_status":"start","queue_duration_s":2.888117,"scheduling_latency_s":2.888144}1980gitlab # [ 530.062735] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:18:07 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/eb0e348b96a80c3e7fb14424c8e4917e98aa1ea8 HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 4591981gitlab # [ 530.065600] gitaly[930]: time="2026-10-02T14:18:07.847Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFN8X2Q4B9N2AJXH1B2DWD diskcache=246ad354-cbde-4ad7-abcc-8f4a99a91057 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-02T20:18:07.836" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:07.836" pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1982gitlab # [ 530.076465] gitaly[930]: time="2026-10-02T14:18:07.848Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=290 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFN8X2Q4B9N2AJXH1B2DWD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-02T20:18:07.836" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:07.836" grpc.time_ms=11.133 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1983gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=727998be4a72c681de6dc36cedfc3d5e9518c280', in 0.10 seconds)1984gitlab # [ 530.090728] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:07.716Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"7b1add2e4d1e1d7df8791624","created_at":"2026-10-02T14:18:04.774Z","trace_propagation_headers":{"sentry-trace":"4e9da5c48d14450db82f1f4a123c4503-75ceaa868373473c","baggage":"sentry-trace_id=4e9da5c48d14450db82f1f4a123c4503,sentry-release=Unknown"},"correlation_id":"01M3YFJ5MGM505KDHE6MBK5E16","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:04.782Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-7b1add2e4d1e1d7df8791624: done: 0.046115 sec","job_status":"done","queue_duration_s":2.888117,"scheduling_latency_s":2.888144,"redis_calls":10,"redis_duration_s":0.00272,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000468,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001707,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000545,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02929,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0004829,"duration_s":0.046115,"completed_at":"2026-10-02T14:18:07.716Z","load_balancing_strategy":"primary","db_duration_s":0.009723,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1985gitlab # [ 530.137906] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:07.718Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"fbbb582668cfa7247a2da743","created_at":"2026-10-02T14:18:04.794Z","trace_propagation_headers":{"sentry-trace":"4e9da5c48d14450db82f1f4a123c4503-75ceaa868373473c","baggage":"sentry-trace_id=4e9da5c48d14450db82f1f4a123c4503,sentry-release=Unknown"},"correlation_id":"01M3YFJ5MGM505KDHE6MBK5E16","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:04.801Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-fbbb582668cfa7247a2da743: start","job_status":"start","queue_duration_s":2.916406,"scheduling_latency_s":2.916435}1986gitlab # [ 530.156355] gitaly[930]: time="2026-10-02T14:18:07.909Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFN8X2Q4B9N2AJXH1B2DWD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:18:37.207" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:07.908" grpc.time_ms=0.957 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1987gitlab # [ 530.170182] nixbot[4119]: INFO: ::ffff:127.0.0.1:35406 - "GET /api/repos/gitlab/root/test-flake/builds?commit=727998be4a72c681de6dc36cedfc3d5e9518c280 HTTP/1.1" 200 OK1988gitlab # [ 530.172373] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:07.804Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"fbbb582668cfa7247a2da743","created_at":"2026-10-02T14:18:04.794Z","trace_propagation_headers":{"sentry-trace":"4e9da5c48d14450db82f1f4a123c4503-75ceaa868373473c","baggage":"sentry-trace_id=4e9da5c48d14450db82f1f4a123c4503,sentry-release=Unknown"},"correlation_id":"01M3YFJ5MGM505KDHE6MBK5E16","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:04.801Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-fbbb582668cfa7247a2da743: done: 0.086855 sec","job_status":"done","queue_duration_s":2.916406,"scheduling_latency_s":2.916435,"redis_calls":14,"redis_duration_s":0.004858,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000465,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002539,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001854,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.053947,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00050674,"duration_s":0.086855,"completed_at":"2026-10-02T14:18:07.804Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016884,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1989gitlab # [ 530.219718] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:07.807Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"0d6280bdee6d3a9b42605ae4","created_at":"2026-10-02T14:18:04.989Z","trace_propagation_headers":{"sentry-trace":"56a8ec8dddf24c3da2a13be0b1207792-1a0151fe36c249b4","baggage":"sentry-trace_id=56a8ec8dddf24c3da2a13be0b1207792,sentry-release=Unknown"},"correlation_id":"01M3YFN8X2Q4B9N2AJXH1B2DWD","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"cb7b72d796f050803acaa952","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:04.992Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-0d6280bdee6d3a9b42605ae4: start","job_status":"start","queue_duration_s":2.815035,"scheduling_latency_s":2.815066}1990gitlab # [ 530.238238] gitaly[930]: time="2026-10-02T14:18:07.923Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFN8X2Q4B9N2AJXH1B2DWD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-02T14:18:17.033" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:18:07.913" grpc.time_ms=9.515 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1991gitlab # [ 530.252540] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:07.875Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"0d6280bdee6d3a9b42605ae4","created_at":"2026-10-02T14:18:04.989Z","trace_propagation_headers":{"sentry-trace":"56a8ec8dddf24c3da2a13be0b1207792-1a0151fe36c249b4","baggage":"sentry-trace_id=56a8ec8dddf24c3da2a13be0b1207792,sentry-release=Unknown"},"correlation_id":"01M3YFN8X2Q4B9N2AJXH1B2DWD","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"cb7b72d796f050803acaa952","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:04.992Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-0d6280bdee6d3a9b42605ae4: done: 0.068387 sec","job_status":"done","queue_duration_s":2.815035,"scheduling_latency_s":2.815066,"gitaly_calls":1,"gitaly_duration_s":0.014316,"redis_calls":12,"redis_duration_s":0.012565999999999999,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003776,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.002787,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.006003,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.008,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.008,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032765,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0370175,"duration_s":0.068387,"completed_at":"2026-10-02T14:18:07.875Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007824,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1992gitlab # [ 530.304723] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:07.878Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"66629b967a034bf238de5040","created_at":"2026-10-02T14:18:05.135Z","trace_propagation_headers":{"sentry-trace":"56a8ec8dddf24c3da2a13be0b1207792-1a0151fe36c249b4","baggage":"sentry-trace_id=56a8ec8dddf24c3da2a13be0b1207792,sentry-release=Unknown"},"correlation_id":"01M3YFN8X2Q4B9N2AJXH1B2DWD","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:52d263650696940c53e04a4d737ef14f367c068fc1f18db5a22ae5d2019cd063","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:05.138Z","job_size_bytes":467,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-66629b967a034bf238de5040: start","job_status":"start","queue_duration_s":2.739911,"scheduling_latency_s":2.739945}1993gitlab # [ 530.324653] gitaly[930]: time="2026-10-02T14:18:07.972Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFN8X2Q4B9N2AJXH1B2DWD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-02T14:18:37.270" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:07.971" grpc.time_ms=1.336 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1994gitlab # [ 530.338902] nixbot[4119]: {"ts": "2026-10-02T14:18:08.225Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/eb0e348b96a80c3e7fb14424c8e4917e98aa1ea8 \"HTTP/1.1 201 Created\""}1995gitlab # [ 530.342702] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:18:08 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/eb0e348b96a80c3e7fb14424c8e4917e98aa1ea8 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 3371996gitlab # [ 530.345651] gitaly[930]: time="2026-10-02T14:18:07.986Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=294 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=6 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFN8X2Q4B9N2AJXH1B2DWD error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-10-02T14:18:37.277" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:07.977" grpc.time_ms=8.605 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1997gitlab # [ 530.364680] gitaly[930]: time="2026-10-02T14:18:08.006Z" level=info msg="finished unary call with code OK" catfile.duration_ms=3 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=3 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNH4EDHAEJ2PTBXY846AX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:18:38.002" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:08.002" grpc.time_ms=3.783 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1998gitlab # [ 530.378894] gitaly[930]: time="2026-10-02T14:18:08.032Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFN8X2Q4B9N2AJXH1B2DWD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T14:18:18.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:18:08.024" grpc.time_ms=7.602 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1999gitlab # [ 530.393765] gitaly[930]: time="2026-10-02T14:18:08.122Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=570 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFNH4EDHAEJ2PTBXY846AX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T14:18:18.101" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:18:08.111" grpc.time_ms=11.009 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2000gitlab # [ 530.435894] gitaly[930]: time="2026-10-02T14:18:08.345Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNHF9S4TR3RP7D7VJRK3C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:18:38.344" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:08.344" grpc.time_ms=0.997 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2001gitlab # [ 530.517843] gitaly[930]: time="2026-10-02T14:18:08.427Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=4 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFNHF9S4TR3RP7D7VJRK3C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T14:18:18.003" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:18:08.414" grpc.time_ms=13.605 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2002gitlab # [ 530.618864] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:18:08 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/eb0e348b96a80c3e7fb14424c8e4917e98aa1ea8 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 2942003gitlab # [ 530.623480] nixbot[4119]: {"ts": "2026-10-02T14:18:08.530Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/eb0e348b96a80c3e7fb14424c8e4917e98aa1ea8 \"HTTP/1.1 201 Created\""}2004gitlab # [ 530.900179] gitaly[930]: time="2026-10-02T14:18:08.810Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=19 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=881 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=3 command.system_time_ms=19 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFN8X2Q4B9N2AJXH1B2DWD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-10-02T14:18:38.083" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:08.784" grpc.time_ms=25.632 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2005gitlab # [ 530.927969] gitaly[930]: time="2026-10-02T14:18:08.838Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=3 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFN8X2Q4B9N2AJXH1B2DWD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-10-02T14:18:38.114" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:08.815" grpc.time_ms=22.467 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2006gitlab # [ 530.951356] sidekiq-cluster[3989]: {"severity":"WARN","time":"2026-10-02T14:18:08.859Z","class":"CreatePipelineWorker","correlation_id":"01M3YFN8X2Q4B9N2AJXH1B2DWD","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"eb0e348b96a80c3e7fb14424c8e4917e98aa1ea8","after":"727998be4a72c681de6dc36cedfc3d5e9518c280","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"727998be4a72c681de6dc36cedfc3d5e9518c280","ref":"refs/heads/master"},"retry":0}2007gitlab # [ 530.963524] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:08.869Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"66629b967a034bf238de5040","created_at":"2026-10-02T14:18:05.135Z","trace_propagation_headers":{"sentry-trace":"56a8ec8dddf24c3da2a13be0b1207792-1a0151fe36c249b4","baggage":"sentry-trace_id=56a8ec8dddf24c3da2a13be0b1207792,sentry-release=Unknown"},"correlation_id":"01M3YFN8X2Q4B9N2AJXH1B2DWD","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:52d263650696940c53e04a4d737ef14f367c068fc1f18db5a22ae5d2019cd063","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:05.138Z","job_size_bytes":467,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-66629b967a034bf238de5040: done: 0.9907 sec","job_status":"done","queue_duration_s":2.739911,"scheduling_latency_s":2.739945,"gitaly_calls":7,"gitaly_duration_s":0.098661,"redis_calls":33,"redis_duration_s":0.010211,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000535,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001763,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002636,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.005277,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.031,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.028,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.860234,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.9907,"completed_at":"2026-10-02T14:18:08.869Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.030795,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2008gitlab # [ 531.025526] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:08.873Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"7ec2571a-b687-44db-ac6f-c8cb70127dd3\"}"],"class":"WebHookWorker","jid":"33ea9f76606583ba2c567021","created_at":"2026-10-02T14:18:05.163Z","trace_propagation_headers":{"sentry-trace":"56a8ec8dddf24c3da2a13be0b1207792-1a0151fe36c249b4","baggage":"sentry-trace_id=56a8ec8dddf24c3da2a13be0b1207792,sentry-release=Unknown"},"correlation_id":"01M3YFN8X2Q4B9N2AJXH1B2DWD","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bec738e951560ad948e7822e0342749b8cc1b9c7cfda55e10e1d955b25dc438a","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:05.168Z","job_size_bytes":1749,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-33ea9f76606583ba2c567021: start","job_status":"start","queue_duration_s":3.704829,"scheduling_latency_s":3.704869}2009gitlab # [ 531.046120] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:08.937Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"7ec2571a-b687-44db-ac6f-c8cb70127dd3\"}"],"class":"WebHookWorker","jid":"33ea9f76606583ba2c567021","created_at":"2026-10-02T14:18:05.163Z","trace_propagation_headers":{"sentry-trace":"56a8ec8dddf24c3da2a13be0b1207792-1a0151fe36c249b4","baggage":"sentry-trace_id=56a8ec8dddf24c3da2a13be0b1207792,sentry-release=Unknown"},"correlation_id":"01M3YFN8X2Q4B9N2AJXH1B2DWD","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bec738e951560ad948e7822e0342749b8cc1b9c7cfda55e10e1d955b25dc438a","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:05.168Z","job_size_bytes":1749,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-33ea9f76606583ba2c567021: done: 0.064415 sec","job_status":"done","queue_duration_s":3.704829,"scheduling_latency_s":3.704869,"redis_calls":14,"redis_duration_s":0.004665,"redis_read_bytes":25,"redis_write_bytes":4685,"redis_queues_calls":2,"redis_queues_duration_s":0.000438,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3729,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002421,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001806,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.012918000000013308,"cpu_s":0.037704,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.064415,"completed_at":"2026-10-02T14:18:08.937Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003936,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2010gitlab # [ 531.097348] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:08.939Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"e3465b3868f9e0306d53a3e2","created_at":"2026-10-02T14:18:05.322Z","trace_propagation_headers":{"sentry-trace":"90767057870b479387804363a8a8b6bd-faa1da711e084327","baggage":"sentry-trace_id=90767057870b479387804363a8a8b6bd,sentry-release=Unknown"},"correlation_id":"01M3YFHZP98CM7CNP4Q3XMNH64","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:05.329Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-e3465b3868f9e0306d53a3e2: start","job_status":"start","queue_duration_s":3.60975,"scheduling_latency_s":3.60978}2011gitlab # [ 531.114651] nixbot[4119]: INFO: ::ffff:127.0.0.1:46172 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted2012gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=727998be4a72c681de6dc36cedfc3d5e9518c280'2013gitlab # [ 531.116047] gitaly[930]: time="2026-10-02T14:18:08.917Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFNHT09QQ7FC11TT4Q47CY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:18:08.902" grpc.time_ms=15.573 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc2014gitlab # [ 531.129540] gitaly[930]: time="2026-10-02T14:18:09.016Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNJ4R5B35V0YHQEQ4ER5Y grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:18:09.003" pid=930 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2015gitlab # [ 531.141657] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:18:08 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 3412016gitlab # [ 531.144090] gitlab-workhorse[816]: time="2026-10-02T14:18:09Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672017gitlab # [ 531.146107] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:18:09 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 972018gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=727998be4a72c681de6dc36cedfc3d5e9518c280', in 0.07 seconds)2019gitlab # [ 531.148249] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:09.005Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"e3465b3868f9e0306d53a3e2","created_at":"2026-10-02T14:18:05.322Z","trace_propagation_headers":{"sentry-trace":"90767057870b479387804363a8a8b6bd-faa1da711e084327","baggage":"sentry-trace_id=90767057870b479387804363a8a8b6bd,sentry-release=Unknown"},"correlation_id":"01M3YFHZP98CM7CNP4Q3XMNH64","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:05.329Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-e3465b3868f9e0306d53a3e2: done: 0.06569 sec","job_status":"done","queue_duration_s":3.60975,"scheduling_latency_s":3.60978,"redis_calls":19,"redis_duration_s":0.004543,"redis_read_bytes":21,"redis_write_bytes":4292,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000867,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000566,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2263,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002286,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000824,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.048675,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.010485,"duration_s":0.06569,"completed_at":"2026-10-02T14:18:09.005Z","load_balancing_strategy":"primary","db_duration_s":0.007803,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2020gitlab # [ 531.198406] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:09.007Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","2143c77444fa7f4a0b86f27a"],"class":"WebHooks::LogExecutionWorker","jid":"4f6e039ae205f6393a42cd2e","created_at":"2026-10-02T14:18:07.029Z","trace_propagation_headers":{"sentry-trace":"90767057870b479387804363a8a8b6bd-125bed3146224efc","baggage":"sentry-trace_id=90767057870b479387804363a8a8b6bd,sentry-release=Unknown"},"correlation_id":"01M3YFHZP98CM7CNP4Q3XMNH64","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdf07dacf65c65a7329e8fe7e044c96605bcd60a4d1c6f47d13627217a8eedc9","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:07.037Z","job_size_bytes":2593,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-4f6e039ae205f6393a42cd2e: start","job_status":"start","queue_duration_s":1.969458,"scheduling_latency_s":1.969488}2021gitlab # [ 531.216955] gitaly[930]: time="2026-10-02T14:18:09.016Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNJ4R5B35V0YHQEQ4ER5Y grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:18:09.003" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc2022gitlab # [ 531.231993] nixbot[4119]: INFO: ::ffff:127.0.0.1:46178 - "GET /api/repos/gitlab/root/test-flake/builds?commit=727998be4a72c681de6dc36cedfc3d5e9518c280 HTTP/1.1" 200 OK2023gitlab # [ 531.234199] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:09.098Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","2143c77444fa7f4a0b86f27a"],"class":"WebHooks::LogExecutionWorker","jid":"4f6e039ae205f6393a42cd2e","created_at":"2026-10-02T14:18:07.029Z","trace_propagation_headers":{"sentry-trace":"90767057870b479387804363a8a8b6bd-125bed3146224efc","baggage":"sentry-trace_id=90767057870b479387804363a8a8b6bd,sentry-release=Unknown"},"correlation_id":"01M3YFHZP98CM7CNP4Q3XMNH64","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdf07dacf65c65a7329e8fe7e044c96605bcd60a4d1c6f47d13627217a8eedc9","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:07.037Z","job_size_bytes":2593,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-4f6e039ae205f6393a42cd2e: done: 0.090897 sec","job_status":"done","queue_duration_s":1.969458,"scheduling_latency_s":1.969488,"redis_calls":10,"redis_duration_s":0.008931,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000836,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.007066,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001029,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.023,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.023,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.044072,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00111246,"duration_s":0.090897,"completed_at":"2026-10-02T14:18:09.098Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.02846,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2024gitlab # [ 531.282639] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:09.105Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"f3167c3deb12973a1c2d36aa","created_at":"2026-10-02T14:18:07.849Z","trace_propagation_headers":{"sentry-trace":"f1913282fe9a409ca7c973e1519f8795-520c4e9301444308","baggage":"sentry-trace_id=f1913282fe9a409ca7c973e1519f8795,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFNGNVJJXYADCQMCTD2X99","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:07.855Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f3167c3deb12973a1c2d36aa: start","job_status":"start","queue_duration_s":1.250023,"scheduling_latency_s":1.250063}2025gitlab # [ 531.299806] gitaly[930]: time="2026-10-02T14:18:09.017Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNJ4R5B35V0YHQEQ4ER5Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:18:09.003" grpc.time_ms=13.596 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc2026gitlab # [ 531.315551] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:18:09 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 932027gitlab # [ 531.317942] gitaly[930]: time="2026-10-02T14:18:09.155Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=4 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M3YFNJ96Y3KM2JVMXWX7B0G7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=206 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:18:09.140" grpc.time_ms=14.596 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc2028gitlab # [ 531.341832] gitaly[930]: time="2026-10-02T14:18:09.251Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNJC73VTQ9WXXQ1BZ8AHB grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:18:09.237" pid=930 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2029gitlab # [ 531.352936] gitaly[930]: time="2026-10-02T14:18:09.251Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNJC73VTQ9WXXQ1BZ8AHB grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:18:09.237" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc2030gitlab # [ 531.367755] gitlab-workhorse[816]: time="2026-10-02T14:18:09Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672031gitlab # [ 531.369974] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:18:09 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 932032gitlab # [ 531.372040] gitaly[930]: time="2026-10-02T14:18:09.252Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNJC73VTQ9WXXQ1BZ8AHB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:18:09.237" grpc.time_ms=14.792 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc2033gitlab # [ 531.387767] gitaly[930]: time="2026-10-02T14:18:09.258Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFNGNVJJXYADCQMCTD2X99 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:18:09.257" grpc.time_ms=0.748 pid=930 span.kind=server system=grpc2034gitlab # [ 531.398257] gitaly[930]: time="2026-10-02T14:18:09.294Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFNGNVJJXYADCQMCTD2X99 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:18:09.293" grpc.time_ms=0.723 pid=930 span.kind=server system=grpc2035gitlab # [ 531.412010] gitaly[930]: time="2026-10-02T14:18:09.322Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFNGNVJJXYADCQMCTD2X99 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:18:09.321" grpc.time_ms=0.509 pid=930 span.kind=server system=grpc2036gitlab # [ 531.425375] gitaly[930]: time="2026-10-02T14:18:09.329Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNGNVJJXYADCQMCTD2X99 diskcache=8f1fd4c3-8061-4199-8e59-b6de881a59ff grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-02T14:18:39.016" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:18:09.217" pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2037gitlab # [ 531.435883] gitaly[930]: time="2026-10-02T14:18:09.330Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=108 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7234 command.oublock=0 command.real_time_ms=107 command.spawn_token_fork_ms=1 command.system_time_ms=58 command.user_time_ms=50 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNGNVJJXYADCQMCTD2X99 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-02T14:18:39.016" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:18:09.217" grpc.time_ms=112.719 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2038gitlab # [ 531.463970] gitaly[930]: time="2026-10-02T14:18:09.374Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M3YFNJGB29ZA28WVJJ823S2Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:18:09.363" grpc.time_ms=10.409 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc2039gitlab # [ 531.478372] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:18:09 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 912040gitlab # [ 531.491961] gitaly[930]: time="2026-10-02T14:18:09.401Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFNGNVJJXYADCQMCTD2X99 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-02T14:18:39.200" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:09.400" grpc.time_ms=1.107 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2041gitlab # [ 531.534730] gitaly[930]: time="2026-10-02T14:18:09.444Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNGNVJJXYADCQMCTD2X99 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:18:39.243" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:09.443" grpc.time_ms=0.813 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2042gitlab # [ 531.550842] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:09.459Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"f3167c3deb12973a1c2d36aa","created_at":"2026-10-02T14:18:07.849Z","trace_propagation_headers":{"sentry-trace":"f1913282fe9a409ca7c973e1519f8795-520c4e9301444308","baggage":"sentry-trace_id=f1913282fe9a409ca7c973e1519f8795,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFNGNVJJXYADCQMCTD2X99","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:07.855Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":3989,"job_status":"rescheduled","message":"PipelineProcessWorker JID-f3167c3deb12973a1c2d36aa: rescheduled"}2043gitlab # [ 531.567729] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:09.471Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"f3167c3deb12973a1c2d36aa","created_at":"2026-10-02T14:18:07.849Z","trace_propagation_headers":{"sentry-trace":"f1913282fe9a409ca7c973e1519f8795-520c4e9301444308","baggage":"sentry-trace_id=f1913282fe9a409ca7c973e1519f8795,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFNGNVJJXYADCQMCTD2X99","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:07.855Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f3167c3deb12973a1c2d36aa: done: 0.366267 sec","job_status":"done","queue_duration_s":1.250023,"scheduling_latency_s":1.250063,"gitaly_calls":3,"gitaly_duration_s":0.124087,"redis_calls":66,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.019419,"redis_read_bytes":148,"redis_write_bytes":18015,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000646,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001099,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.004049,"redis_queues_read_bytes":26,"redis_queues_write_bytes":12314,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.008791,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000275,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000468,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.003445,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.036,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.009,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.009,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.019,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.18792,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.29235228,"duration_s":0.366267,"completed_at":"2026-10-02T14:18:09.471Z","load_balancing_strategy":"primary","db_duration_s":0.037543,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2044gitlab # [ 531.624557] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:09.474Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","33ea9f76606583ba2c567021"],"class":"WebHooks::LogExecutionWorker","jid":"31db6a9195e5de01c7d8d85e","created_at":"2026-10-02T14:18:08.924Z","trace_propagation_headers":{"sentry-trace":"56a8ec8dddf24c3da2a13be0b1207792-4ebaf45163ae41a1","baggage":"sentry-trace_id=56a8ec8dddf24c3da2a13be0b1207792,sentry-release=Unknown"},"correlation_id":"01M3YFN8X2Q4B9N2AJXH1B2DWD","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:28b55b202858ce7195c10cb64e5029716cd617be9d04d1068a90834f70bc8c67","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:08.928Z","job_size_bytes":2592,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-31db6a9195e5de01c7d8d85e: start","job_status":"start","queue_duration_s":0.546266,"scheduling_latency_s":0.546303}2045gitlab # [ 531.642384] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:09.516Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","33ea9f76606583ba2c567021"],"class":"WebHooks::LogExecutionWorker","jid":"31db6a9195e5de01c7d8d85e","created_at":"2026-10-02T14:18:08.924Z","trace_propagation_headers":{"sentry-trace":"56a8ec8dddf24c3da2a13be0b1207792-4ebaf45163ae41a1","baggage":"sentry-trace_id=56a8ec8dddf24c3da2a13be0b1207792,sentry-release=Unknown"},"correlation_id":"01M3YFN8X2Q4B9N2AJXH1B2DWD","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:28b55b202858ce7195c10cb64e5029716cd617be9d04d1068a90834f70bc8c67","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:08.928Z","job_size_bytes":2592,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-31db6a9195e5de01c7d8d85e: done: 0.041386 sec","job_status":"done","queue_duration_s":0.546266,"scheduling_latency_s":0.546303,"redis_calls":7,"redis_duration_s":0.002345,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00171,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000635,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028234,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.041386,"completed_at":"2026-10-02T14:18:09.516Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005947,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2046gitlab # [ 531.688389] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:09.518Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"54aa076076528a59ddd11157","created_at":"2026-10-02T14:18:08.986Z","trace_propagation_headers":{"sentry-trace":"90767057870b479387804363a8a8b6bd-4ee5e52775044b43","baggage":"sentry-trace_id=90767057870b479387804363a8a8b6bd,sentry-release=Unknown"},"correlation_id":"01M3YFHZP98CM7CNP4Q3XMNH64","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:08.993Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-54aa076076528a59ddd11157: start","job_status":"start","queue_duration_s":0.524932,"scheduling_latency_s":0.524965}2047gitlab # [ 531.706371] gitlab-workhorse[816]: time="2026-10-02T14:18:09Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2162048gitlab # [ 531.708370] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:18:09 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 1002049gitlab # [ 531.710379] gitaly[930]: time="2026-10-02T14:18:09.484Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNJK9GHYGWVVVRMBG3NB4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:18:09.466" pid=930 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc2050gitlab # [ 531.721294] gitaly[930]: time="2026-10-02T14:18:09.484Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1103 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNJK9GHYGWVVVRMBG3NB4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:18:09.466" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc2051gitlab # [ 531.736412] gitaly[930]: time="2026-10-02T14:18:09.486Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1103 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNJK9GHYGWVVVRMBG3NB4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:18:09.466" grpc.time_ms=19.27 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc2052gitlab # [ 531.752540] gitaly[930]: time="2026-10-02T14:18:09.645Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNJQVDHGNTMFPQDSKQ74M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:18:39.643" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:09.644" grpc.time_ms=0.742 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2053gitlab # [ 531.766290] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:09.659Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"54aa076076528a59ddd11157","created_at":"2026-10-02T14:18:08.986Z","trace_propagation_headers":{"sentry-trace":"90767057870b479387804363a8a8b6bd-4ee5e52775044b43","baggage":"sentry-trace_id=90767057870b479387804363a8a8b6bd,sentry-release=Unknown"},"correlation_id":"01M3YFHZP98CM7CNP4Q3XMNH64","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:08.993Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-54aa076076528a59ddd11157: done: 0.14034 sec","job_status":"done","queue_duration_s":0.524932,"scheduling_latency_s":0.524965,"redis_calls":11,"redis_duration_s":0.00356,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000506,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002505,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000549,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.012,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.029,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.090693,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00042952,"duration_s":0.14034,"completed_at":"2026-10-02T14:18:09.658Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.047607,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2054gitlab # [ 531.814381] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:09.661Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"2706652d9f37684b6e6f3d0f","created_at":"2026-10-02T14:18:09.206Z","trace_propagation_headers":{"sentry-trace":"f1913282fe9a409ca7c973e1519f8795-87e828dc5954419a","baggage":"sentry-trace_id=f1913282fe9a409ca7c973e1519f8795,sentry-release=Unknown"},"correlation_id":"01M3YFNGNVJJXYADCQMCTD2X99","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:09.209Z","job_size_bytes":23,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-2706652d9f37684b6e6f3d0f: start","job_status":"start","queue_duration_s":0.45219,"scheduling_latency_s":0.452226}2055gitlab # [ 531.833178] gitaly[930]: time="2026-10-02T14:18:09.669Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=592 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M3YFNJQVDHGNTMFPQDSKQ74M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-10-02T14:18:39.657" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:18:09.658" grpc.time_ms=11.59 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2056gitlab # [ 531.848333] gitaly[930]: time="2026-10-02T14:18:09.691Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=5 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFNJQVDHGNTMFPQDSKQ74M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T14:18:19.219" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:18:09.679" grpc.time_ms=11.721 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2057gitlab # [ 531.862774] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:09.745Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"2706652d9f37684b6e6f3d0f","created_at":"2026-10-02T14:18:09.206Z","trace_propagation_headers":{"sentry-trace":"f1913282fe9a409ca7c973e1519f8795-87e828dc5954419a","baggage":"sentry-trace_id=f1913282fe9a409ca7c973e1519f8795,sentry-release=Unknown"},"correlation_id":"01M3YFNGNVJJXYADCQMCTD2X99","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:09.209Z","job_size_bytes":23,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-2706652d9f37684b6e6f3d0f: done: 0.08419 sec","job_status":"done","queue_duration_s":0.45219,"scheduling_latency_s":0.452226,"redis_calls":7,"redis_duration_s":0.0019260000000000002,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001373,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000553,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.05818,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.08419,"completed_at":"2026-10-02T14:18:09.745Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.019754,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2058gitlab # [ 531.907056] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:09.749Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"13493f3446fd406aa3c91daa","created_at":"2026-10-02T14:18:09.335Z","trace_propagation_headers":{"sentry-trace":"f1913282fe9a409ca7c973e1519f8795-87e828dc5954419a","baggage":"sentry-trace_id=f1913282fe9a409ca7c973e1519f8795,sentry-release=Unknown"},"correlation_id":"01M3YFNGNVJJXYADCQMCTD2X99","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:09.338Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-13493f3446fd406aa3c91daa: start","job_status":"start","queue_duration_s":0.410245,"scheduling_latency_s":0.410283}2059gitlab # [ 531.925026] gitaly[930]: time="2026-10-02T14:18:09.731Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNJQVDHGNTMFPQDSKQ74M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-02T14:18:19.270" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:18:09.731" grpc.time_ms=0.308 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2060gitlab # [ 531.936808] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:09.801Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"13493f3446fd406aa3c91daa","created_at":"2026-10-02T14:18:09.335Z","trace_propagation_headers":{"sentry-trace":"f1913282fe9a409ca7c973e1519f8795-87e828dc5954419a","baggage":"sentry-trace_id=f1913282fe9a409ca7c973e1519f8795,sentry-release=Unknown"},"correlation_id":"01M3YFNGNVJJXYADCQMCTD2X99","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:09.338Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-13493f3446fd406aa3c91daa: done: 0.052583 sec","job_status":"done","queue_duration_s":0.410245,"scheduling_latency_s":0.410283,"redis_calls":7,"redis_duration_s":0.002349,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00167,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000679,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039619,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.052583,"completed_at":"2026-10-02T14:18:09.801Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006407,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2061gitlab # [ 531.982841] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:09.803Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"b18e0fd21e47c944bd92c79d","created_at":"2026-10-02T14:18:09.347Z","trace_propagation_headers":{"sentry-trace":"f1913282fe9a409ca7c973e1519f8795-87e828dc5954419a","baggage":"sentry-trace_id=f1913282fe9a409ca7c973e1519f8795,sentry-release=Unknown"},"correlation_id":"01M3YFNGNVJJXYADCQMCTD2X99","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:09.352Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-b18e0fd21e47c944bd92c79d: start","job_status":"start","queue_duration_s":0.451472,"scheduling_latency_s":0.451513}2062gitlab # [ 532.000188] gitaly[930]: time="2026-10-02T14:18:09.750Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=570 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNJQVDHGNTMFPQDSKQ74M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-02T14:18:19.279" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:18:09.740" grpc.time_ms=9.878 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2063gitlab # [ 532.014464] gitaly[930]: time="2026-10-02T14:18:09.792Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFNJQVDHGNTMFPQDSKQ74M grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4154 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:18:09.791" grpc.time_ms=0.396 pid=930 span.kind=server system=grpc2064gitlab # [ 532.026788] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:09.851Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"b18e0fd21e47c944bd92c79d","created_at":"2026-10-02T14:18:09.347Z","trace_propagation_headers":{"sentry-trace":"f1913282fe9a409ca7c973e1519f8795-87e828dc5954419a","baggage":"sentry-trace_id=f1913282fe9a409ca7c973e1519f8795,sentry-release=Unknown"},"correlation_id":"01M3YFNGNVJJXYADCQMCTD2X99","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:09.352Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-b18e0fd21e47c944bd92c79d: done: 0.048519 sec","job_status":"done","queue_duration_s":0.451472,"scheduling_latency_s":0.451513,"redis_calls":7,"redis_duration_s":0.0018999999999999998,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001468,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000432,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033904,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.048519,"completed_at":"2026-10-02T14:18:09.851Z","load_balancing_strategy":"primary","db_duration_s":0.007576,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2065gitlab # [ 532.071204] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:09.853Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"c135e044b1e9b621d3d68db6","created_at":"2026-10-02T14:18:09.360Z","trace_propagation_headers":{"sentry-trace":"f1913282fe9a409ca7c973e1519f8795-87e828dc5954419a","baggage":"sentry-trace_id=f1913282fe9a409ca7c973e1519f8795,sentry-release=Unknown"},"correlation_id":"01M3YFNGNVJJXYADCQMCTD2X99","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:09.364Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-c135e044b1e9b621d3d68db6: start","job_status":"start","queue_duration_s":0.489191,"scheduling_latency_s":0.489222}2066gitlab # [ 532.087419] gitaly[930]: time="2026-10-02T14:18:09.822Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFNJQVDHGNTMFPQDSKQ74M grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4156 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:18:09.822" grpc.time_ms=0.492 pid=930 span.kind=server system=grpc2067gitlab # [ 532.097439] gitaly[930]: time="2026-10-02T14:18:09.831Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNJQVDHGNTMFPQDSKQ74M diskcache=5a86e52f-4b50-4f2c-915b-9e1a9541aef5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-02T14:18:19.003" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:18:09.753" pid=930 ref=refs/keep-around/727998be4a72c681de6dc36cedfc3d5e9518c280 remote_ip="2001:db8:1::1" revision=727998be4a72c681de6dc36cedfc3d5e9518c280 span.kind=server system=grpc username=root2068gitlab # [ 532.109228] nixbot[4119]: {"ts": "2026-10-02T14:18:09.975Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/727998be4a72c681de6dc36cedfc3d5e9518c280 \"HTTP/1.1 201 Created\""}2069gitlab # [ 532.112090] nixbot[4119]: {"ts": "2026-10-02T14:18:09.988Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-727998be4a72-5"}2070gitlab # [ 532.114468] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:09.910Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"c135e044b1e9b621d3d68db6","created_at":"2026-10-02T14:18:09.360Z","trace_propagation_headers":{"sentry-trace":"f1913282fe9a409ca7c973e1519f8795-87e828dc5954419a","baggage":"sentry-trace_id=f1913282fe9a409ca7c973e1519f8795,sentry-release=Unknown"},"correlation_id":"01M3YFNGNVJJXYADCQMCTD2X99","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:09.364Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-c135e044b1e9b621d3d68db6: done: 0.056467 sec","job_status":"done","queue_duration_s":0.489191,"scheduling_latency_s":0.489222,"redis_calls":10,"redis_duration_s":0.013663,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00848,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00179,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003393,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032722,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.056467,"completed_at":"2026-10-02T14:18:09.910Z","load_balancing_strategy":"primary","db_duration_s":0.007375,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2071gitlab # [ 532.159515] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:09.913Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"80d5df2c195a0e9557893369","created_at":"2026-10-02T14:18:09.365Z","trace_propagation_headers":{"sentry-trace":"f1913282fe9a409ca7c973e1519f8795-87e828dc5954419a","baggage":"sentry-trace_id=f1913282fe9a409ca7c973e1519f8795,sentry-release=Unknown"},"correlation_id":"01M3YFNGNVJJXYADCQMCTD2X99","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:09.368Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-80d5df2c195a0e9557893369: start","job_status":"start","queue_duration_s":0.544926,"scheduling_latency_s":0.544961}2072gitlab # [ 532.176730] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:18:09 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/727998be4a72c681de6dc36cedfc3d5e9518c280 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 4422073gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=727998be4a72c681de6dc36cedfc3d5e9518c280'2074gitlab # [ 532.179296] gitaly[930]: time="2026-10-02T14:18:09.831Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=70 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5232 command.oublock=8 command.real_time_ms=73 command.spawn_token_fork_ms=1 command.system_time_ms=45 command.user_time_ms=25 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNJQVDHGNTMFPQDSKQ74M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-02T14:18:19.003" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:18:09.753" grpc.time_ms=77.536 pid=930 ref=refs/keep-around/727998be4a72c681de6dc36cedfc3d5e9518c280 remote_ip="2001:db8:1::1" revision=727998be4a72c681de6dc36cedfc3d5e9518c280 span.kind=server system=grpc username=root2075gitlab # [ 532.194431] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:09.967Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"80d5df2c195a0e9557893369","created_at":"2026-10-02T14:18:09.365Z","trace_propagation_headers":{"sentry-trace":"f1913282fe9a409ca7c973e1519f8795-87e828dc5954419a","baggage":"sentry-trace_id=f1913282fe9a409ca7c973e1519f8795,sentry-release=Unknown"},"correlation_id":"01M3YFNGNVJJXYADCQMCTD2X99","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:09.368Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-80d5df2c195a0e9557893369: done: 0.054108 sec","job_status":"done","queue_duration_s":0.544926,"scheduling_latency_s":0.544961,"redis_calls":7,"redis_duration_s":0.006799,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006157,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000642,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033001,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.054108,"completed_at":"2026-10-02T14:18:09.967Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007976,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2076gitlab # [ 532.238789] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:09.970Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"936122335d918f742da31b28","created_at":"2026-10-02T14:18:09.369Z","trace_propagation_headers":{"sentry-trace":"f1913282fe9a409ca7c973e1519f8795-87e828dc5954419a","baggage":"sentry-trace_id=f1913282fe9a409ca7c973e1519f8795,sentry-release=Unknown"},"correlation_id":"01M3YFNGNVJJXYADCQMCTD2X99","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:09.373Z","job_size_bytes":19,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-936122335d918f742da31b28: start","job_status":"start","queue_duration_s":0.596984,"scheduling_latency_s":0.597019}2077gitlab # [ 532.256371] gitaly[930]: time="2026-10-02T14:18:09.836Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNJQVDHGNTMFPQDSKQ74M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:18:39.835" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:09.836" grpc.time_ms=0.5 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2078gitlab # [ 532.269910] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:10.005Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"936122335d918f742da31b28","created_at":"2026-10-02T14:18:09.369Z","trace_propagation_headers":{"sentry-trace":"f1913282fe9a409ca7c973e1519f8795-87e828dc5954419a","baggage":"sentry-trace_id=f1913282fe9a409ca7c973e1519f8795,sentry-release=Unknown"},"correlation_id":"01M3YFNGNVJJXYADCQMCTD2X99","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:09.373Z","job_size_bytes":19,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-936122335d918f742da31b28: done: 0.035082 sec","job_status":"done","queue_duration_s":0.596984,"scheduling_latency_s":0.597019,"redis_calls":8,"redis_duration_s":0.002143,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001604,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000539,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023512,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.035082,"completed_at":"2026-10-02T14:18:10.004Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004119,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2079gitlab # [ 532.315053] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:10.007Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"a3042e4d5b4566019563a204","created_at":"2026-10-02T14:18:09.374Z","trace_propagation_headers":{"sentry-trace":"f1913282fe9a409ca7c973e1519f8795-87e828dc5954419a","baggage":"sentry-trace_id=f1913282fe9a409ca7c973e1519f8795,sentry-release=Unknown"},"correlation_id":"01M3YFNGNVJJXYADCQMCTD2X99","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:09.377Z","job_size_bytes":66,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-a3042e4d5b4566019563a204: start","job_status":"start","queue_duration_s":0.63049,"scheduling_latency_s":0.630553}2080gitlab # [ 532.334008] nixbot[4119]: {"ts": "2026-10-02T14:18:10.167Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-727998be4a72-5", "argv0": "bwrap"}2081gitlab # [ 532.336703] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:10.048Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"a3042e4d5b4566019563a204","created_at":"2026-10-02T14:18:09.374Z","trace_propagation_headers":{"sentry-trace":"f1913282fe9a409ca7c973e1519f8795-87e828dc5954419a","baggage":"sentry-trace_id=f1913282fe9a409ca7c973e1519f8795,sentry-release=Unknown"},"correlation_id":"01M3YFNGNVJJXYADCQMCTD2X99","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:09.377Z","job_size_bytes":66,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-a3042e4d5b4566019563a204: done: 0.040486 sec","job_status":"done","queue_duration_s":0.63049,"scheduling_latency_s":0.630553,"redis_calls":7,"redis_duration_s":0.0018670000000000002,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001378,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000489,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025074,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.040486,"completed_at":"2026-10-02T14:18:10.048Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007678,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2082gitlab # [ 532.382937] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:10.050Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"83bf199c069deeb33678a385","created_at":"2026-10-02T14:18:09.378Z","trace_propagation_headers":{"sentry-trace":"f1913282fe9a409ca7c973e1519f8795-87e828dc5954419a","baggage":"sentry-trace_id=f1913282fe9a409ca7c973e1519f8795,sentry-release=Unknown"},"correlation_id":"01M3YFNGNVJJXYADCQMCTD2X99","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:09.380Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-83bf199c069deeb33678a385: start","job_status":"start","queue_duration_s":0.670254,"scheduling_latency_s":0.670285}2083gitlab # [ 532.400504] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:10.085Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"83bf199c069deeb33678a385","created_at":"2026-10-02T14:18:09.378Z","trace_propagation_headers":{"sentry-trace":"f1913282fe9a409ca7c973e1519f8795-87e828dc5954419a","baggage":"sentry-trace_id=f1913282fe9a409ca7c973e1519f8795,sentry-release=Unknown"},"correlation_id":"01M3YFNGNVJJXYADCQMCTD2X99","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:09.380Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-83bf199c069deeb33678a385: done: 0.034493 sec","job_status":"done","queue_duration_s":0.670254,"scheduling_latency_s":0.670285,"redis_calls":7,"redis_duration_s":0.00185,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001315,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000535,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023298,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034493,"completed_at":"2026-10-02T14:18:10.085Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005052,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2084gitlab # [ 532.447234] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:10.087Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"df114c28d70e5530dbbe18d1","created_at":"2026-10-02T14:18:09.381Z","trace_propagation_headers":{"sentry-trace":"f1913282fe9a409ca7c973e1519f8795-87e828dc5954419a","baggage":"sentry-trace_id=f1913282fe9a409ca7c973e1519f8795,sentry-release=Unknown"},"correlation_id":"01M3YFNGNVJJXYADCQMCTD2X99","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:09.384Z","job_size_bytes":6,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-df114c28d70e5530dbbe18d1: start","job_status":"start","queue_duration_s":0.702629,"scheduling_latency_s":0.702659}2085gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=727998be4a72c681de6dc36cedfc3d5e9518c280', in 0.28 seconds)2086(finished: subtest: push delivers the webhook, in 113.67 seconds)2087subtest: eval and build post commit statuses2088gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/727998be4a72c681de6dc36cedfc3d5e9518c280/statuses2089gitlab # [ 532.464356] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:10.108Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"df114c28d70e5530dbbe18d1","created_at":"2026-10-02T14:18:09.381Z","trace_propagation_headers":{"sentry-trace":"f1913282fe9a409ca7c973e1519f8795-87e828dc5954419a","baggage":"sentry-trace_id=f1913282fe9a409ca7c973e1519f8795,sentry-release=Unknown"},"correlation_id":"01M3YFNGNVJJXYADCQMCTD2X99","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:09.384Z","job_size_bytes":6,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-df114c28d70e5530dbbe18d1: done: 0.021661 sec","job_status":"done","queue_duration_s":0.702629,"scheduling_latency_s":0.702659,"redis_calls":7,"redis_duration_s":0.001823,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001319,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000504,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014407,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021661,"completed_at":"2026-10-02T14:18:10.108Z","load_balancing_strategy":"primary","db_duration_s":0.000637,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2090gitlab # [ 532.509772] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:10.110Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"6876064655e23c3f3c484937","created_at":"2026-10-02T14:18:09.385Z","trace_propagation_headers":{"sentry-trace":"f1913282fe9a409ca7c973e1519f8795-87e828dc5954419a","baggage":"sentry-trace_id=f1913282fe9a409ca7c973e1519f8795,sentry-release=Unknown"},"correlation_id":"01M3YFNGNVJJXYADCQMCTD2X99","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:09.387Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-6876064655e23c3f3c484937: start","job_status":"start","queue_duration_s":0.723018,"scheduling_latency_s":0.723048}2091gitlab # [ 532.526313] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:10.149Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"6876064655e23c3f3c484937","created_at":"2026-10-02T14:18:09.385Z","trace_propagation_headers":{"sentry-trace":"f1913282fe9a409ca7c973e1519f8795-87e828dc5954419a","baggage":"sentry-trace_id=f1913282fe9a409ca7c973e1519f8795,sentry-release=Unknown"},"correlation_id":"01M3YFNGNVJJXYADCQMCTD2X99","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:09.387Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-6876064655e23c3f3c484937: done: 0.038694 sec","job_status":"done","queue_duration_s":0.723018,"scheduling_latency_s":0.723048,"redis_calls":7,"redis_duration_s":0.002091,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001594,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000497,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023306,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.038694,"completed_at":"2026-10-02T14:18:10.149Z","load_balancing_strategy":"primary","db_duration_s":0.009139,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2092gitlab # [ 532.569940] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:10.155Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"0dd6c0166debc168b453c88a","created_at":"2026-10-02T14:18:09.460Z","trace_propagation_headers":{"sentry-trace":"f1913282fe9a409ca7c973e1519f8795-87e828dc5954419a","baggage":"sentry-trace_id=f1913282fe9a409ca7c973e1519f8795,sentry-release=Unknown"},"correlation_id":"01M3YFNGNVJJXYADCQMCTD2X99","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:09.463Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-0dd6c0166debc168b453c88a: start","job_status":"start","queue_duration_s":0.692625,"scheduling_latency_s":0.692659}2093gitlab # [ 532.586166] nixbot[4119]: INFO: ::ffff:127.0.0.1:46194 - "GET /api/repos/gitlab/root/test-flake/builds?commit=727998be4a72c681de6dc36cedfc3d5e9518c280 HTTP/1.1" 200 OK2094gitlab # [ 532.588572] gitaly[930]: time="2026-10-02T14:18:10.470Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFNJQVDHGNTMFPQDSKQ74M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-02T14:18:40.267" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:10.468" grpc.time_ms=1.835 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2095gitlab # [ 532.601651] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:10.200Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"0dd6c0166debc168b453c88a","created_at":"2026-10-02T14:18:09.460Z","trace_propagation_headers":{"sentry-trace":"f1913282fe9a409ca7c973e1519f8795-87e828dc5954419a","baggage":"sentry-trace_id=f1913282fe9a409ca7c973e1519f8795,sentry-release=Unknown"},"correlation_id":"01M3YFNGNVJJXYADCQMCTD2X99","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:09.463Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-0dd6c0166debc168b453c88a: done: 0.044459 sec","job_status":"done","queue_duration_s":0.692625,"scheduling_latency_s":0.692659,"redis_calls":8,"redis_duration_s":0.002153,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001663,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00049,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028606,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.044459,"completed_at":"2026-10-02T14:18:10.200Z","load_balancing_strategy":"primary","db_duration_s":0.007872,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2096gitlab # [ 532.647897] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:10.202Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"0d6fdd9bc6c10a9edbbf4d79","created_at":"2026-10-02T14:18:09.843Z","trace_propagation_headers":{"sentry-trace":"be83e3703ceb4ec8929e59f6b9a24942-07d84effd20546ed","baggage":"sentry-trace_id=be83e3703ceb4ec8929e59f6b9a24942,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFNJQVDHGNTMFPQDSKQ74M","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:09.849Z","job_size_bytes":65,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-0d6fdd9bc6c10a9edbbf4d79: start","job_status":"start","queue_duration_s":0.353076,"scheduling_latency_s":0.353107}2097gitlab # [ 532.665874] nix-daemon[4288]: accepted connection from pid 5500, user nixbot2098gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/727998be4a72c681de6dc36cedfc3d5e9518c280/statuses, in 0.20 seconds)2099gitlab # [ 532.666929] gitaly[930]: time="2026-10-02T14:18:10.529Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNKKNY3H0X0VCHXE7HFJR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:18:40.527" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:10.528" grpc.time_ms=0.871 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2100gitlab # [ 532.680162] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:10.244Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"0d6fdd9bc6c10a9edbbf4d79","created_at":"2026-10-02T14:18:09.843Z","trace_propagation_headers":{"sentry-trace":"be83e3703ceb4ec8929e59f6b9a24942-07d84effd20546ed","baggage":"sentry-trace_id=be83e3703ceb4ec8929e59f6b9a24942,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFNJQVDHGNTMFPQDSKQ74M","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:09.849Z","job_size_bytes":65,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-0d6fdd9bc6c10a9edbbf4d79: done: 0.042214 sec","job_status":"done","queue_duration_s":0.353076,"scheduling_latency_s":0.353107,"redis_calls":7,"redis_duration_s":0.0017859999999999998,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001256,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00053,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030652,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.042214,"completed_at":"2026-10-02T14:18:10.244Z","load_balancing_strategy":"primary","db_duration_s":0.005329,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2101gitlab # [ 532.724829] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:10.246Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"1f3d2120ea11eb124c84e14d","created_at":"2026-10-02T14:18:09.851Z","trace_propagation_headers":{"sentry-trace":"be83e3703ceb4ec8929e59f6b9a24942-07d84effd20546ed","baggage":"sentry-trace_id=be83e3703ceb4ec8929e59f6b9a24942,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFNJQVDHGNTMFPQDSKQ74M","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:09.856Z","job_size_bytes":65,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-1f3d2120ea11eb124c84e14d: start","job_status":"start","queue_duration_s":0.39031,"scheduling_latency_s":0.390342}2102gitlab # [ 532.743382] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:18:10 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/727998be4a72c681de6dc36cedfc3d5e9518c280/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1562103gitlab # [ 532.746375] gitaly[930]: time="2026-10-02T14:18:10.543Z" level=info msg="finished unary call with code OK" catfile.duration_ms=3 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=3 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNJQVDHGNTMFPQDSKQ74M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:18:40.338" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:10.539" grpc.time_ms=3.574 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2104gitlab # [ 532.759899] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:10.321Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"1f3d2120ea11eb124c84e14d","created_at":"2026-10-02T14:18:09.851Z","trace_propagation_headers":{"sentry-trace":"be83e3703ceb4ec8929e59f6b9a24942-07d84effd20546ed","baggage":"sentry-trace_id=be83e3703ceb4ec8929e59f6b9a24942,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFNJQVDHGNTMFPQDSKQ74M","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:09.856Z","job_size_bytes":65,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-1f3d2120ea11eb124c84e14d: done: 0.075206 sec","job_status":"done","queue_duration_s":0.39031,"scheduling_latency_s":0.390342,"redis_calls":10,"redis_duration_s":0.003416,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002079,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.001337,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.050942,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.075206,"completed_at":"2026-10-02T14:18:10.321Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017575,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2105gitlab # [ 532.805667] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:10.323Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"fd0d33cb9e18002b13294142","created_at":"2026-10-02T14:18:09.945Z","trace_propagation_headers":{"sentry-trace":"be83e3703ceb4ec8929e59f6b9a24942-07d84effd20546ed","baggage":"sentry-trace_id=be83e3703ceb4ec8929e59f6b9a24942,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFNJQVDHGNTMFPQDSKQ74M","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:09.952Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-fd0d33cb9e18002b13294142: start","job_status":"start","queue_duration_s":0.371517,"scheduling_latency_s":0.371552}2106gitlab # [ 532.822907] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:10.571Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"fd0d33cb9e18002b13294142","created_at":"2026-10-02T14:18:09.945Z","trace_propagation_headers":{"sentry-trace":"be83e3703ceb4ec8929e59f6b9a24942-07d84effd20546ed","baggage":"sentry-trace_id=be83e3703ceb4ec8929e59f6b9a24942,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFNJQVDHGNTMFPQDSKQ74M","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:09.952Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-fd0d33cb9e18002b13294142: done: 0.247466 sec","job_status":"done","queue_duration_s":0.371517,"scheduling_latency_s":0.371552,"gitaly_calls":2,"gitaly_duration_s":0.011771,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.020544,"redis_read_bytes":79,"redis_write_bytes":5121,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.00166,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.007277,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.00049,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1132,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.005903,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000394,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.001013,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002147,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.046,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.009,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.013,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.022,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.17269,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.19480414,"duration_s":0.247466,"completed_at":"2026-10-02T14:18:10.570Z","load_balancing_strategy":"primary","db_duration_s":0.052446,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2107gitlab # [ 532.879349] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:10.573Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"207bbf5314eacbb876a0fbd6","created_at":"2026-10-02T14:18:10.429Z","trace_propagation_headers":{"sentry-trace":"be83e3703ceb4ec8929e59f6b9a24942-831efb5445f34154","baggage":"sentry-trace_id=be83e3703ceb4ec8929e59f6b9a24942,sentry-release=Unknown"},"correlation_id":"01M3YFNJQVDHGNTMFPQDSKQ74M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:10.435Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-207bbf5314eacbb876a0fbd6: start","job_status":"start","queue_duration_s":0.138095,"scheduling_latency_s":0.138134}2108gitlab # [ 532.913534] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:10.823Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"207bbf5314eacbb876a0fbd6","created_at":"2026-10-02T14:18:10.429Z","trace_propagation_headers":{"sentry-trace":"be83e3703ceb4ec8929e59f6b9a24942-831efb5445f34154","baggage":"sentry-trace_id=be83e3703ceb4ec8929e59f6b9a24942,sentry-release=Unknown"},"correlation_id":"01M3YFNJQVDHGNTMFPQDSKQ74M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:10.435Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-207bbf5314eacbb876a0fbd6: done: 0.249665 sec","job_status":"done","queue_duration_s":0.138095,"scheduling_latency_s":0.138134,"redis_calls":7,"redis_duration_s":0.001991,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001433,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000558,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.234787,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.249665,"completed_at":"2026-10-02T14:18:10.822Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006605,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2109gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/727998be4a72c681de6dc36cedfc3d5e9518c280/statuses2110gitlab # [ 533.820293] gitaly[930]: time="2026-10-02T14:18:11.729Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNMSA40FS13WW9GK61DEV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:18:41.727" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:11.728" grpc.time_ms=0.936 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2111gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/727998be4a72c681de6dc36cedfc3d5e9518c280/statuses, in 0.20 seconds)2112gitlab # [ 533.869794] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:18:11 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/727998be4a72c681de6dc36cedfc3d5e9518c280/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1522113gitlab # [ 534.457641] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:12.365Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"9ea81c91696bcab01089ad46","created_at":"2026-10-02T14:18:12.341Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a486b05ceaba5042fe5b2ecf6327508b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:12.361Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-9ea81c91696bcab01089ad46: start","job_status":"start","queue_duration_s":0.004214,"scheduling_latency_s":0.004258}2114gitlab # [ 534.525479] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:12.435Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}2115gitlab # [ 534.590003] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:12.499Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"9ea81c91696bcab01089ad46","created_at":"2026-10-02T14:18:12.341Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a486b05ceaba5042fe5b2ecf6327508b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:12.361Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-9ea81c91696bcab01089ad46: done: 0.133091 sec","job_status":"done","queue_duration_s":0.004214,"scheduling_latency_s":0.004258,"redis_calls":9,"redis_duration_s":0.016367,"redis_read_bytes":7,"redis_write_bytes":1178,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.006137,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006182,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004048,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02993,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.133091,"completed_at":"2026-10-02T14:18:12.498Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007138,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2116gitlab # [ 534.657449] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:12.503Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"f144040edee502c954d59a72","created_at":"2026-10-02T14:18:12.381Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"42aea1f11675c6ca2cab3b9d57af8031","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:12.395Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-f144040edee502c954d59a72: start","job_status":"start","queue_duration_s":0.107945,"scheduling_latency_s":0.107989}2117gitlab # [ 534.675385] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:12.572Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"f144040edee502c954d59a72","created_at":"2026-10-02T14:18:12.381Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"42aea1f11675c6ca2cab3b9d57af8031","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:12.395Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-f144040edee502c954d59a72: done: 0.06854 sec","job_status":"done","queue_duration_s":0.107945,"scheduling_latency_s":0.107989,"redis_calls":11,"redis_duration_s":0.012531,"redis_read_bytes":9,"redis_write_bytes":1386,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.005532,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005636,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001363,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031457,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.06854,"completed_at":"2026-10-02T14:18:12.571Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003641,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2118gitlab # [ 534.722442] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:12.580Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"54c313eac1db2bded1d46fbf","created_at":"2026-10-02T14:18:12.419Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cfc028658a435dedd5913ff091caca5f","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:12.444Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-54c313eac1db2bded1d46fbf: start","job_status":"start","queue_duration_s":0.135848,"scheduling_latency_s":0.135891}2119gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/727998be4a72c681de6dc36cedfc3d5e9518c280/statuses2120gitlab # [ 534.831122] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:12.738Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"54c313eac1db2bded1d46fbf","created_at":"2026-10-02T14:18:12.419Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cfc028658a435dedd5913ff091caca5f","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:12.444Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-54c313eac1db2bded1d46fbf: done: 0.158629 sec","job_status":"done","queue_duration_s":0.135848,"scheduling_latency_s":0.135891,"redis_calls":11,"redis_duration_s":0.022607,"redis_read_bytes":8,"redis_write_bytes":1094,"redis_cache_calls":2,"redis_cache_duration_s":0.004355,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003573,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.010341,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":482,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004338,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033824,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.158629,"completed_at":"2026-10-02T14:18:12.738Z","load_balancing_strategy":"primary","db_duration_s":0.008244,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2121gitlab # [ 534.900106] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:12.747Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"73de38343503d85446408ec7","created_at":"2026-10-02T14:18:12.485Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d8a052dc8c2475305f182ef2f5094afc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:12.542Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-73de38343503d85446408ec7: start","job_status":"start","queue_duration_s":0.204972,"scheduling_latency_s":0.205019}2122gitlab # [ 535.022926] gitaly[930]: time="2026-10-02T14:18:12.932Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNNZDWZRE8SXRQHQR7G34 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:18:42.730" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:12.931" grpc.time_ms=0.88 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2123gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/727998be4a72c681de6dc36cedfc3d5e9518c280/statuses, in 0.20 seconds)2124gitlab # [ 535.064920] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:12.974Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"73de38343503d85446408ec7","created_at":"2026-10-02T14:18:12.485Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d8a052dc8c2475305f182ef2f5094afc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:12.542Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-73de38343503d85446408ec7: done: 0.226883 sec","job_status":"done","queue_duration_s":0.204972,"scheduling_latency_s":0.205019,"redis_calls":13,"redis_duration_s":0.024118,"redis_read_bytes":11,"redis_write_bytes":1780,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00955,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.007896,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.006672,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.038,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.023,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.081118,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00341742,"exclusive_lock_hold_duration_s":0.09799834,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.226883,"completed_at":"2026-10-02T14:18:12.974Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.047999,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2125gitlab # [ 535.121965] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:12.980Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"002548b8d459c40d35d2e379","created_at":"2026-10-02T14:18:12.561Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e1986666840acb98bdc1bc652f0d4d20","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-02T14:18:12.577Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-002548b8d459c40d35d2e379: start","job_status":"start","queue_duration_s":0.40238,"scheduling_latency_s":0.402416}2126gitlab # [ 535.137284] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:13.029Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}2127gitlab # [ 535.141737] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:18:12 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/727998be4a72c681de6dc36cedfc3d5e9518c280/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1342128gitlab # [ 535.160300] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:13.069Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"002548b8d459c40d35d2e379","created_at":"2026-10-02T14:18:12.561Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e1986666840acb98bdc1bc652f0d4d20","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-02T14:18:12.577Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-002548b8d459c40d35d2e379: done: 0.089567 sec","job_status":"done","queue_duration_s":0.40238,"scheduling_latency_s":0.402416,"redis_calls":9,"redis_duration_s":0.013974,"redis_read_bytes":7,"redis_write_bytes":1152,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002264,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006848,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004862,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022773,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.089567,"completed_at":"2026-10-02T14:18:13.069Z","load_balancing_strategy":"primary","db_duration_s":0.005769,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2129gitlab # [ 535.206832] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:13.079Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"6228683f760b33e8841e56d9","created_at":"2026-10-02T14:18:12.627Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4df91d3a84b1ef452dc33d43f91a1600","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:12.636Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-6228683f760b33e8841e56d9: start","job_status":"start","queue_duration_s":0.442639,"scheduling_latency_s":0.442677}2130gitlab # [ 535.316132] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:13.225Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"6228683f760b33e8841e56d9","created_at":"2026-10-02T14:18:12.627Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4df91d3a84b1ef452dc33d43f91a1600","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:12.636Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-6228683f760b33e8841e56d9: done: 0.146159 sec","job_status":"done","queue_duration_s":0.442639,"scheduling_latency_s":0.442677,"redis_calls":9,"redis_duration_s":0.012759000000000001,"redis_read_bytes":7,"redis_write_bytes":984,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001832,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.008023,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002904,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031546,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.146159,"completed_at":"2026-10-02T14:18:13.225Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011598,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2131gitlab # [ 535.382140] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:13.233Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"086666839ee7d02d04bf2200","created_at":"2026-10-02T14:18:12.651Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:12.673Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-086666839ee7d02d04bf2200: start","job_status":"start","queue_duration_s":0.5593,"scheduling_latency_s":0.559334}2132gitlab # [ 535.566315] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:13.475Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"086666839ee7d02d04bf2200","created_at":"2026-10-02T14:18:12.651Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:12.673Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-086666839ee7d02d04bf2200: done: 0.242371 sec","job_status":"done","queue_duration_s":0.5593,"scheduling_latency_s":0.559334,"redis_calls":70,"redis_duration_s":0.034485999999999996,"redis_read_bytes":118,"redis_write_bytes":42480,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.012376,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000681,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37385,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.017171,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.004258,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.037,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.037,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.154314,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.242371,"completed_at":"2026-10-02T14:18:13.475Z","load_balancing_strategy":"primary","db_duration_s":0.036461,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2133gitlab # [ 535.635036] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:13.477Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"4eb3528cb99ab9ce66aa1137","created_at":"2026-10-02T14:18:12.695Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bd4e40ceeb0ec71cfa4918a3263a7a03","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-02T14:18:12.713Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-4eb3528cb99ab9ce66aa1137: start","job_status":"start","queue_duration_s":0.763982,"scheduling_latency_s":0.76401}2134gitlab # [ 535.652409] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:13.491Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}2135gitlab # [ 535.656255] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:13.500Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"4eb3528cb99ab9ce66aa1137","created_at":"2026-10-02T14:18:12.695Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bd4e40ceeb0ec71cfa4918a3263a7a03","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-02T14:18:12.713Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-4eb3528cb99ab9ce66aa1137: done: 0.022958 sec","job_status":"done","queue_duration_s":0.763982,"scheduling_latency_s":0.76401,"redis_calls":9,"redis_duration_s":0.002026,"redis_read_bytes":7,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00042,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001132,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000474,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013919,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022958,"completed_at":"2026-10-02T14:18:13.500Z","load_balancing_strategy":"primary","db_duration_s":0.000741,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2136gitlab # [ 535.700671] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:13.501Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"7887349cdc05c426dc5e1a56","created_at":"2026-10-02T14:18:12.733Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"03512bb630a8d24103309ed8e00e4673","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:12.753Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-7887349cdc05c426dc5e1a56: start","job_status":"start","queue_duration_s":0.748834,"scheduling_latency_s":0.748862}2137gitlab # [ 535.900324] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:13.809Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"7887349cdc05c426dc5e1a56","created_at":"2026-10-02T14:18:12.733Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"03512bb630a8d24103309ed8e00e4673","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:12.753Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-7887349cdc05c426dc5e1a56: done: 0.307925 sec","job_status":"done","queue_duration_s":0.748834,"scheduling_latency_s":0.748862,"redis_calls":13,"redis_duration_s":0.003165,"redis_read_bytes":11,"redis_write_bytes":1567,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000849,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001223,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001093,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.045,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.045,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.24921,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00066246,"exclusive_lock_hold_duration_s":0.27941844,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.307925,"completed_at":"2026-10-02T14:18:13.809Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.067131,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2138gitlab # [ 535.975745] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:13.811Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"afb26e4234a497e21c48c12d","created_at":"2026-10-02T14:18:12.765Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1735e4f0ef9223ff4a67349e2997fb52","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:12.789Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-afb26e4234a497e21c48c12d: start","job_status":"start","queue_duration_s":1.021966,"scheduling_latency_s":1.021995}2139gitlab # [ 535.995288] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:13.848Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"afb26e4234a497e21c48c12d","created_at":"2026-10-02T14:18:12.765Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1735e4f0ef9223ff4a67349e2997fb52","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:12.789Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-afb26e4234a497e21c48c12d: done: 0.036858 sec","job_status":"done","queue_duration_s":1.021966,"scheduling_latency_s":1.021995,"redis_calls":11,"redis_duration_s":0.003125,"redis_read_bytes":9,"redis_write_bytes":1399,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000535,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001386,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001204,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022966,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0007689,"exclusive_lock_hold_duration_s":0.0101082,"duration_s":0.036858,"completed_at":"2026-10-02T14:18:13.848Z","load_balancing_strategy":"primary","db_duration_s":0.004888,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2140gitlab # [ 536.051340] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:13.850Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"9e51cc4239054770f7ccb312","created_at":"2026-10-02T14:18:12.804Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bc1ee80365a44312af5090c6f0285918","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:12.825Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-9e51cc4239054770f7ccb312: start","job_status":"start","queue_duration_s":1.024932,"scheduling_latency_s":1.024963}2141gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/727998be4a72c681de6dc36cedfc3d5e9518c280/statuses2142gitlab # [ 536.068683] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:13.885Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"9e51cc4239054770f7ccb312","created_at":"2026-10-02T14:18:12.804Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bc1ee80365a44312af5090c6f0285918","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:12.825Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-9e51cc4239054770f7ccb312: done: 0.035402 sec","job_status":"done","queue_duration_s":1.024932,"scheduling_latency_s":1.024963,"redis_calls":11,"redis_duration_s":0.003408,"redis_read_bytes":9,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000504,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001348,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001556,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021984,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00080896,"exclusive_lock_hold_duration_s":0.00694432,"duration_s":0.035402,"completed_at":"2026-10-02T14:18:13.885Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003545,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2143gitlab # [ 536.117898] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:13.888Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"97e1ed894fcbd866d41b6bb1","created_at":"2026-10-02T14:18:12.841Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0fb701d0cba412818d66c73aecd0a301","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:12.876Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-97e1ed894fcbd866d41b6bb1: start","job_status":"start","queue_duration_s":1.011304,"scheduling_latency_s":1.011337}2144gitlab # [ 536.135357] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:13.951Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"97e1ed894fcbd866d41b6bb1","created_at":"2026-10-02T14:18:12.841Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0fb701d0cba412818d66c73aecd0a301","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:12.876Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-97e1ed894fcbd866d41b6bb1: done: 0.063486 sec","job_status":"done","queue_duration_s":1.011304,"scheduling_latency_s":1.011337,"redis_calls":11,"redis_duration_s":0.003954,"redis_read_bytes":9,"redis_write_bytes":1518,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000559,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001669,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001726,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034932,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00145066,"exclusive_lock_hold_duration_s":0.01968596,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.063486,"completed_at":"2026-10-02T14:18:13.951Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.019606,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2145gitlab # [ 536.185353] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:13.953Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"43fbbef7df72013bdb631d1b","created_at":"2026-10-02T14:18:12.890Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7e74bb0a698019197eda090841f73f6c","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:12.916Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-43fbbef7df72013bdb631d1b: start","job_status":"start","queue_duration_s":1.037267,"scheduling_latency_s":1.037303}2146gitlab # [ 536.233587] gitaly[930]: time="2026-10-02T14:18:14.142Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNQ4X9CVSSZHB19N86JMP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:18:44.140" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:14.141" grpc.time_ms=1.226 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2147gitlab # [ 536.280401] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:18:14 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/727998be4a72c681de6dc36cedfc3d5e9518c280/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1432148gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/727998be4a72c681de6dc36cedfc3d5e9518c280/statuses, in 0.21 seconds)2149gitlab # [ 536.599952] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:14.508Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"43fbbef7df72013bdb631d1b","created_at":"2026-10-02T14:18:12.890Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7e74bb0a698019197eda090841f73f6c","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:12.916Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-43fbbef7df72013bdb631d1b: done: 0.554852 sec","job_status":"done","queue_duration_s":1.037267,"scheduling_latency_s":1.037303,"redis_calls":564,"redis_duration_s":0.14110899999999998,"redis_read_bytes":562,"redis_write_bytes":49063,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000665,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001606,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.138838,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.483676,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.554852,"completed_at":"2026-10-02T14:18:14.508Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000812,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2150gitlab # [ 536.669261] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:14.510Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"e6da7476114f48e3cf1790c1","created_at":"2026-10-02T14:18:12.930Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2431f51d8e96d937bfea154a2a4879c7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:12.952Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-e6da7476114f48e3cf1790c1: start","job_status":"start","queue_duration_s":1.558876,"scheduling_latency_s":1.558913}2151gitlab # [ 536.688734] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:14.542Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"e6da7476114f48e3cf1790c1","created_at":"2026-10-02T14:18:12.930Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2431f51d8e96d937bfea154a2a4879c7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:12.952Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-e6da7476114f48e3cf1790c1: done: 0.031685 sec","job_status":"done","queue_duration_s":1.558876,"scheduling_latency_s":1.558913,"redis_calls":9,"redis_duration_s":0.002827,"redis_read_bytes":7,"redis_write_bytes":1192,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000669,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001556,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000602,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019537,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031685,"completed_at":"2026-10-02T14:18:14.542Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001044,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2152gitlab # [ 536.737178] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:14.544Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"63a41c678d41cd7e89aa6aa3","created_at":"2026-10-02T14:18:12.967Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"78d73303ac1d23b22f6e9de2d5f02200","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:12.983Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-63a41c678d41cd7e89aa6aa3: start","job_status":"start","queue_duration_s":1.561569,"scheduling_latency_s":1.561605}2153gitlab # [ 536.830156] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:14.739Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"63a41c678d41cd7e89aa6aa3","created_at":"2026-10-02T14:18:12.967Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"78d73303ac1d23b22f6e9de2d5f02200","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:12.983Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-63a41c678d41cd7e89aa6aa3: done: 0.194929 sec","job_status":"done","queue_duration_s":1.561569,"scheduling_latency_s":1.561605,"redis_calls":12,"redis_duration_s":0.003898,"redis_read_bytes":11,"redis_write_bytes":1247,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00057,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.0017,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001628,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":296,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.178926,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00064408,"extra.object_storage_delete_stale_direct_uploads_worker.total_pending_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.total_deleted_stale_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.execution_timeout":false,"duration_s":0.194929,"completed_at":"2026-10-02T14:18:14.739Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000648,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2154gitlab # [ 536.902372] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:14.742Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"b5d8f33718cbf1395170468d","created_at":"2026-10-02T14:18:12.992Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"095070b3fd8feef0289b0f6684746413","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.010Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-b5d8f33718cbf1395170468d: start","job_status":"start","queue_duration_s":1.731994,"scheduling_latency_s":1.732028}2155gitlab # [ 536.926119] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:14.762Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}2156gitlab # [ 536.931780] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:14.775Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"b5d8f33718cbf1395170468d","created_at":"2026-10-02T14:18:12.992Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"095070b3fd8feef0289b0f6684746413","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.010Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-b5d8f33718cbf1395170468d: done: 0.033509 sec","job_status":"done","queue_duration_s":1.731994,"scheduling_latency_s":1.732028,"redis_calls":9,"redis_duration_s":0.002966,"redis_read_bytes":7,"redis_write_bytes":1112,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00066,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00165,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000656,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020572,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033509,"completed_at":"2026-10-02T14:18:14.775Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000817,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2157gitlab # [ 536.992338] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:14.777Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"9f56fc8d46fb12304487a04f","created_at":"2026-10-02T14:18:13.021Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5b1590e3b489d546af90eb6a10f6b1bb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.039Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-9f56fc8d46fb12304487a04f: start","job_status":"start","queue_duration_s":1.738001,"scheduling_latency_s":1.738028}2158gitlab # [ 537.009493] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:14.791Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}2159gitlab # [ 537.013557] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:14.799Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"9f56fc8d46fb12304487a04f","created_at":"2026-10-02T14:18:13.021Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5b1590e3b489d546af90eb6a10f6b1bb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.039Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-9f56fc8d46fb12304487a04f: done: 0.022281 sec","job_status":"done","queue_duration_s":1.738001,"scheduling_latency_s":1.738028,"redis_calls":9,"redis_duration_s":0.002195,"redis_read_bytes":7,"redis_write_bytes":1105,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000472,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001326,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000397,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013331,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022281,"completed_at":"2026-10-02T14:18:14.799Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000713,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2160gitlab # [ 537.059957] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:14.801Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"8ac1ff07188b74654d6bfd29","created_at":"2026-10-02T14:18:13.063Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f95dec89b4e4f9c283ff8a1eb71e0886","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.077Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-8ac1ff07188b74654d6bfd29: start","job_status":"start","queue_duration_s":1.723548,"scheduling_latency_s":1.723564}2161gitlab # [ 537.075468] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:14.834Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"8ac1ff07188b74654d6bfd29","created_at":"2026-10-02T14:18:13.063Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f95dec89b4e4f9c283ff8a1eb71e0886","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.077Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-8ac1ff07188b74654d6bfd29: done: 0.03312 sec","job_status":"done","queue_duration_s":1.723548,"scheduling_latency_s":1.723564,"redis_calls":11,"redis_duration_s":0.002944,"redis_read_bytes":9,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000442,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001356,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001146,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019257,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00065364,"exclusive_lock_hold_duration_s":0.00404434,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.03312,"completed_at":"2026-10-02T14:18:14.834Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002374,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2162gitlab # [ 537.122267] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:14.835Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"f509229009faa18970e39e94","created_at":"2026-10-02T14:18:13.123Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8acf832943dc29d3c91ed530e57c5102","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.132Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-f509229009faa18970e39e94: start","job_status":"start","queue_duration_s":1.702545,"scheduling_latency_s":1.70256}2163gitlab # [ 537.137261] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:14.873Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"f509229009faa18970e39e94","created_at":"2026-10-02T14:18:13.123Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8acf832943dc29d3c91ed530e57c5102","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.132Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-f509229009faa18970e39e94: done: 0.038174 sec","job_status":"done","queue_duration_s":1.702545,"scheduling_latency_s":1.70256,"redis_calls":9,"redis_duration_s":0.002092,"redis_read_bytes":7,"redis_write_bytes":1183,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000428,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001187,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000477,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018139,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.038174,"completed_at":"2026-10-02T14:18:14.873Z","load_balancing_strategy":"primary","db_duration_s":0.012479,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2164gitlab # [ 537.182949] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:14.875Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"cf9fe7db0b64d6c53cf54e61","created_at":"2026-10-02T14:18:13.150Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bb136775b0afba9d7ed0c3d9a57ee69e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.171Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-cf9fe7db0b64d6c53cf54e61: start","job_status":"start","queue_duration_s":1.703809,"scheduling_latency_s":1.703826}2165gitlab # [ 537.197600] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:14.918Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"cf9fe7db0b64d6c53cf54e61","created_at":"2026-10-02T14:18:13.150Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bb136775b0afba9d7ed0c3d9a57ee69e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.171Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-cf9fe7db0b64d6c53cf54e61: done: 0.042943 sec","job_status":"done","queue_duration_s":1.703809,"scheduling_latency_s":1.703826,"redis_calls":18,"redis_duration_s":0.003963,"redis_read_bytes":20,"redis_write_bytes":4702,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000917,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000501,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001824,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000721,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032871,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.042943,"completed_at":"2026-10-02T14:18:14.918Z","load_balancing_strategy":"primary","db_duration_s":0.003027,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2166gitlab # [ 537.244667] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:14.920Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"16330d77a4816bd0c223a404","created_at":"2026-10-02T14:18:13.183Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a3a6f31b983610ad09a211b978a0b1f8","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.203Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-16330d77a4816bd0c223a404: start","job_status":"start","queue_duration_s":1.717309,"scheduling_latency_s":1.717327}2167gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/727998be4a72c681de6dc36cedfc3d5e9518c280/statuses2168gitlab # [ 537.259748] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:14.957Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"16330d77a4816bd0c223a404","created_at":"2026-10-02T14:18:13.183Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a3a6f31b983610ad09a211b978a0b1f8","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.203Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-16330d77a4816bd0c223a404: done: 0.037182 sec","job_status":"done","queue_duration_s":1.717309,"scheduling_latency_s":1.717327,"redis_calls":9,"redis_duration_s":0.002255,"redis_read_bytes":7,"redis_write_bytes":1026,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00038,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001349,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000526,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021956,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.037182,"completed_at":"2026-10-02T14:18:14.957Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00566,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2169gitlab # [ 537.304252] nix-daemon[4288]: accepted connection from pid 5534, user nixbot2170gitlab # [ 537.305337] nix-daemon[4288]: accepted connection from pid 5500, user nixbot2171gitlab # [ 537.306390] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:14.959Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"654298626fda45fa0c5506db","created_at":"2026-10-02T14:18:13.218Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5c738792bc2da69c6e51db2d21d24c2d","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.230Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-654298626fda45fa0c5506db: start","job_status":"start","queue_duration_s":1.729569,"scheduling_latency_s":1.729594}2172gitlab # [ 537.322060] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.000Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"654298626fda45fa0c5506db","created_at":"2026-10-02T14:18:13.218Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5c738792bc2da69c6e51db2d21d24c2d","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.230Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-654298626fda45fa0c5506db: done: 0.04068 sec","job_status":"done","queue_duration_s":1.729569,"scheduling_latency_s":1.729594,"redis_calls":9,"redis_duration_s":0.002409,"redis_read_bytes":7,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00055,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001372,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000487,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026287,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.04068,"completed_at":"2026-10-02T14:18:15.000Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004739,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2173gitlab # [ 537.367353] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.002Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"160fc3f99d94b6d6194221b3","created_at":"2026-10-02T14:18:13.242Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"711630c703bc278bb7da8e66cd997d2b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-02T14:18:13.258Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-160fc3f99d94b6d6194221b3: start","job_status":"start","queue_duration_s":1.744239,"scheduling_latency_s":1.744265}2174gitlab # [ 537.381635] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.054Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"160fc3f99d94b6d6194221b3","created_at":"2026-10-02T14:18:13.242Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"711630c703bc278bb7da8e66cd997d2b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-02T14:18:13.258Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-160fc3f99d94b6d6194221b3: done: 0.051821 sec","job_status":"done","queue_duration_s":1.744239,"scheduling_latency_s":1.744265,"redis_calls":9,"redis_duration_s":0.002807,"redis_read_bytes":7,"redis_write_bytes":1068,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000446,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001632,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000729,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032257,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.051821,"completed_at":"2026-10-02T14:18:15.054Z","load_balancing_strategy":"primary","db_duration_s":0.006358,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2175gitlab # [ 537.427051] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.056Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"785be00d0552cd185c389c98","created_at":"2026-10-02T14:18:13.288Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e703d860d5a1eeea2b04689bb501cb26","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.313Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-785be00d0552cd185c389c98: start","job_status":"start","queue_duration_s":1.742664,"scheduling_latency_s":1.742688}2176gitlab # [ 537.442643] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.072Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}2177gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/727998be4a72c681de6dc36cedfc3d5e9518c280/statuses, in 0.20 seconds)2178gitlab # [ 537.446220] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.082Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"785be00d0552cd185c389c98","created_at":"2026-10-02T14:18:13.288Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e703d860d5a1eeea2b04689bb501cb26","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.313Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-785be00d0552cd185c389c98: done: 0.026487 sec","job_status":"done","queue_duration_s":1.742664,"scheduling_latency_s":1.742688,"redis_calls":9,"redis_duration_s":0.0021449999999999998,"redis_read_bytes":7,"redis_write_bytes":1185,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00052,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001194,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000431,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016359,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026487,"completed_at":"2026-10-02T14:18:15.082Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000692,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2179gitlab # [ 537.490252] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.084Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"2d0c1c872281a0edb9a965bf","created_at":"2026-10-02T14:18:13.327Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7d60cb43e75d84a00a90f4e91158dc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.345Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-2d0c1c872281a0edb9a965bf: start","job_status":"start","queue_duration_s":1.738446,"scheduling_latency_s":1.738467}2180gitlab # [ 537.505808] gitaly[930]: time="2026-10-02T14:18:15.344Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNRA9BW9D249QGG6WEZ3B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:18:45.341" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:15.342" grpc.time_ms=1.255 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2181gitlab # [ 537.519461] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:18:15 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/727998be4a72c681de6dc36cedfc3d5e9518c280/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1492182gitlab # [ 537.522448] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.127Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"2d0c1c872281a0edb9a965bf","created_at":"2026-10-02T14:18:13.327Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-443992d4ee94421c","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7d60cb43e75d84a00a90f4e91158dc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.345Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-2d0c1c872281a0edb9a965bf: done: 0.043356 sec","job_status":"done","queue_duration_s":1.738446,"scheduling_latency_s":1.738467,"redis_calls":12,"redis_duration_s":0.00428,"redis_read_bytes":10,"redis_write_bytes":1576,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000471,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001609,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.0022,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02884,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01258434,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.043356,"completed_at":"2026-10-02T14:18:15.127Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00503,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2183gitlab # [ 537.569717] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.131Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"03268c3db965e7503dd50d41","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-03268c3db965e7503dd50d41: start","job_status":"start","queue_duration_s":1.668167,"scheduling_latency_s":1.668194}2184gitlab # [ 537.583103] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.169Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"03268c3db965e7503dd50d41","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-03268c3db965e7503dd50d41: done: 0.038019 sec","job_status":"done","queue_duration_s":1.668167,"scheduling_latency_s":1.668194,"redis_calls":15,"redis_duration_s":0.003223,"redis_read_bytes":12,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000529,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00093,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001764,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020934,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.038019,"completed_at":"2026-10-02T14:18:15.169Z","load_balancing_strategy":"primary","db_duration_s":0.000825,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2185gitlab # [ 537.628397] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.171Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"702586e7e6f9ab486e38aa5a","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-702586e7e6f9ab486e38aa5a: start","job_status":"start","queue_duration_s":1.708416,"scheduling_latency_s":1.708441}2186gitlab # [ 537.641723] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.194Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"702586e7e6f9ab486e38aa5a","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-702586e7e6f9ab486e38aa5a: done: 0.022433 sec","job_status":"done","queue_duration_s":1.708416,"scheduling_latency_s":1.708441,"redis_calls":13,"redis_duration_s":0.0032050000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001096,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002109,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015021,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022433,"completed_at":"2026-10-02T14:18:15.194Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2187gitlab # [ 537.683877] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.197Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dc5051581f468fa7199d2f26","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dc5051581f468fa7199d2f26: start","job_status":"start","queue_duration_s":1.733879,"scheduling_latency_s":1.733907}2188gitlab # [ 537.697389] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.226Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dc5051581f468fa7199d2f26","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dc5051581f468fa7199d2f26: done: 0.029456 sec","job_status":"done","queue_duration_s":1.733879,"scheduling_latency_s":1.733907,"redis_calls":13,"redis_duration_s":0.008137,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003107,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00503,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017106,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.029456,"completed_at":"2026-10-02T14:18:15.226Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2189gitlab # [ 537.739237] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.228Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d091f852bc4fba9b9ff92633","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d091f852bc4fba9b9ff92633: start","job_status":"start","queue_duration_s":1.765182,"scheduling_latency_s":1.765207}2190gitlab # [ 537.752723] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.254Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d091f852bc4fba9b9ff92633","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d091f852bc4fba9b9ff92633: done: 0.025455 sec","job_status":"done","queue_duration_s":1.765182,"scheduling_latency_s":1.765207,"redis_calls":13,"redis_duration_s":0.003713,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001562,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002151,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01559,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025455,"completed_at":"2026-10-02T14:18:15.253Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2191gitlab # [ 537.794465] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.257Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"828245b52a6a67cbca46de57","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-828245b52a6a67cbca46de57: start","job_status":"start","queue_duration_s":1.794002,"scheduling_latency_s":1.794032}2192gitlab # [ 537.807813] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.282Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"828245b52a6a67cbca46de57","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-828245b52a6a67cbca46de57: done: 0.024798 sec","job_status":"done","queue_duration_s":1.794002,"scheduling_latency_s":1.794032,"redis_calls":13,"redis_duration_s":0.003581,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001147,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002434,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016345,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024798,"completed_at":"2026-10-02T14:18:15.282Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2193gitlab # [ 537.850581] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.284Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8b60e90a37e8d74edb68d1ba","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8b60e90a37e8d74edb68d1ba: start","job_status":"start","queue_duration_s":1.82071,"scheduling_latency_s":1.820736}2194gitlab # [ 537.863956] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.308Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8b60e90a37e8d74edb68d1ba","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8b60e90a37e8d74edb68d1ba: done: 0.024252 sec","job_status":"done","queue_duration_s":1.82071,"scheduling_latency_s":1.820736,"redis_calls":13,"redis_duration_s":0.0028780000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001131,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001747,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015464,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024252,"completed_at":"2026-10-02T14:18:15.308Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2195gitlab # [ 537.905861] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.311Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fa65d7dbfb47b646972fc128","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fa65d7dbfb47b646972fc128: start","job_status":"start","queue_duration_s":1.848269,"scheduling_latency_s":1.848298}2196gitlab # [ 537.919225] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.337Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fa65d7dbfb47b646972fc128","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fa65d7dbfb47b646972fc128: done: 0.025712 sec","job_status":"done","queue_duration_s":1.848269,"scheduling_latency_s":1.848298,"redis_calls":13,"redis_duration_s":0.004946,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002787,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002159,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016403,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025712,"completed_at":"2026-10-02T14:18:15.337Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2197gitlab # [ 537.961450] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.340Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fdbbd1a5f31ed757ca01b49f","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fdbbd1a5f31ed757ca01b49f: start","job_status":"start","queue_duration_s":1.876738,"scheduling_latency_s":1.876767}2198gitlab # [ 537.974831] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.372Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fdbbd1a5f31ed757ca01b49f","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fdbbd1a5f31ed757ca01b49f: done: 0.032848 sec","job_status":"done","queue_duration_s":1.876738,"scheduling_latency_s":1.876767,"redis_calls":13,"redis_duration_s":0.011373,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.005239,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006134,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017095,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.032848,"completed_at":"2026-10-02T14:18:15.372Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2199gitlab # [ 538.017101] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.375Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f1d190bf443da40763d232eb","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f1d190bf443da40763d232eb: start","job_status":"start","queue_duration_s":1.912042,"scheduling_latency_s":1.912074}2200gitlab # [ 538.030387] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.403Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f1d190bf443da40763d232eb","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f1d190bf443da40763d232eb: done: 0.02803 sec","job_status":"done","queue_duration_s":1.912042,"scheduling_latency_s":1.912074,"redis_calls":13,"redis_duration_s":0.005618,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001184,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004434,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0165,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02803,"completed_at":"2026-10-02T14:18:15.403Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2201gitlab # [ 538.072985] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.406Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0075640f5347b665210eff56","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0075640f5347b665210eff56: start","job_status":"start","queue_duration_s":1.942598,"scheduling_latency_s":1.942629}2202gitlab # [ 538.085838] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.429Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0075640f5347b665210eff56","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0075640f5347b665210eff56: done: 0.023102 sec","job_status":"done","queue_duration_s":1.942598,"scheduling_latency_s":1.942629,"redis_calls":13,"redis_duration_s":0.002587,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000935,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001652,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015312,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023102,"completed_at":"2026-10-02T14:18:15.428Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2203gitlab # [ 538.126390] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.430Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fb047eb8fb32c16ea0e2ff13","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fb047eb8fb32c16ea0e2ff13: start","job_status":"start","queue_duration_s":1.967521,"scheduling_latency_s":1.967551}2204gitlab # [ 538.139786] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.453Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fb047eb8fb32c16ea0e2ff13","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fb047eb8fb32c16ea0e2ff13: done: 0.022479 sec","job_status":"done","queue_duration_s":1.967521,"scheduling_latency_s":1.967551,"redis_calls":13,"redis_duration_s":0.002818,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000975,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001843,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014865,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022479,"completed_at":"2026-10-02T14:18:15.453Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2205gitlab # [ 538.181729] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.455Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6b9fc46a6a25b85f4a3f8c2","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6b9fc46a6a25b85f4a3f8c2: start","job_status":"start","queue_duration_s":1.992264,"scheduling_latency_s":1.992296}2206gitlab # [ 538.195034] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.478Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6b9fc46a6a25b85f4a3f8c2","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6b9fc46a6a25b85f4a3f8c2: done: 0.023234 sec","job_status":"done","queue_duration_s":1.992264,"scheduling_latency_s":1.992296,"redis_calls":13,"redis_duration_s":0.002869,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001019,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00185,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015207,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023234,"completed_at":"2026-10-02T14:18:15.478Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2207gitlab # [ 538.237149] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.480Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3d85b3b474cd79109834190f","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3d85b3b474cd79109834190f: start","job_status":"start","queue_duration_s":2.017281,"scheduling_latency_s":2.01731}2208gitlab # [ 538.250547] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.504Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3d85b3b474cd79109834190f","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3d85b3b474cd79109834190f: done: 0.024112 sec","job_status":"done","queue_duration_s":2.017281,"scheduling_latency_s":2.01731,"redis_calls":13,"redis_duration_s":0.0033750000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000979,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002396,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015781,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024112,"completed_at":"2026-10-02T14:18:15.504Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2209gitlab # [ 538.293703] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.506Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a4364a1ae9ca4ce409871c58","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a4364a1ae9ca4ce409871c58: start","job_status":"start","queue_duration_s":2.043403,"scheduling_latency_s":2.043428}2210gitlab # [ 538.307104] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.535Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a4364a1ae9ca4ce409871c58","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a4364a1ae9ca4ce409871c58: done: 0.028891 sec","job_status":"done","queue_duration_s":2.043403,"scheduling_latency_s":2.043428,"redis_calls":13,"redis_duration_s":0.004232,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001304,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002928,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020212,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028891,"completed_at":"2026-10-02T14:18:15.535Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2211gitlab # [ 538.348355] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.538Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fcba9b81fb85cb6f616f6ce2","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fcba9b81fb85cb6f616f6ce2: start","job_status":"start","queue_duration_s":2.074625,"scheduling_latency_s":2.07466}2212gitlab # [ 538.361129] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.567Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fcba9b81fb85cb6f616f6ce2","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fcba9b81fb85cb6f616f6ce2: done: 0.029947 sec","job_status":"done","queue_duration_s":2.074625,"scheduling_latency_s":2.07466,"redis_calls":13,"redis_duration_s":0.0033439999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001207,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002137,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019721,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.029947,"completed_at":"2026-10-02T14:18:15.567Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2213gitlab # [ 538.401755] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.570Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5d6a834960e0bdea0ac58225","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5d6a834960e0bdea0ac58225: start","job_status":"start","queue_duration_s":2.106781,"scheduling_latency_s":2.106814}2214gitlab # [ 538.414694] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.597Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5d6a834960e0bdea0ac58225","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5d6a834960e0bdea0ac58225: done: 0.027467 sec","job_status":"done","queue_duration_s":2.106781,"scheduling_latency_s":2.106814,"redis_calls":13,"redis_duration_s":0.0035730000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0015,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002073,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018059,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.027467,"completed_at":"2026-10-02T14:18:15.597Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2215gitlab # [ 538.455523] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.599Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef11544c31edc657bf6c8a00","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef11544c31edc657bf6c8a00: start","job_status":"start","queue_duration_s":2.136332,"scheduling_latency_s":2.136364}2216gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/727998be4a72c681de6dc36cedfc3d5e9518c280/statuses2217gitlab # [ 538.468511] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.625Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef11544c31edc657bf6c8a00","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef11544c31edc657bf6c8a00: done: 0.025532 sec","job_status":"done","queue_duration_s":2.136332,"scheduling_latency_s":2.136364,"redis_calls":13,"redis_duration_s":0.003053,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001049,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002004,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016789,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025532,"completed_at":"2026-10-02T14:18:15.625Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2218gitlab # [ 538.508551] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.627Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"49559aa78844dde7dae27da9","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-49559aa78844dde7dae27da9: start","job_status":"start","queue_duration_s":2.163859,"scheduling_latency_s":2.163892}2219gitlab # [ 538.521411] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.652Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"49559aa78844dde7dae27da9","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-49559aa78844dde7dae27da9: done: 0.025036 sec","job_status":"done","queue_duration_s":2.163859,"scheduling_latency_s":2.163892,"redis_calls":13,"redis_duration_s":0.0029509999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000995,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001956,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016552,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025036,"completed_at":"2026-10-02T14:18:15.652Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2220gitlab # [ 538.562067] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.654Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3a5968450592be0cc0545ac0","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3a5968450592be0cc0545ac0: start","job_status":"start","queue_duration_s":2.190849,"scheduling_latency_s":2.19088}2221gitlab # [ 538.574769] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.677Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3a5968450592be0cc0545ac0","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3a5968450592be0cc0545ac0: done: 0.023724 sec","job_status":"done","queue_duration_s":2.190849,"scheduling_latency_s":2.19088,"redis_calls":13,"redis_duration_s":0.00284,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000985,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001855,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015801,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023724,"completed_at":"2026-10-02T14:18:15.677Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2222gitlab # [ 538.615360] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.679Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d206ed9c5e22e16cf6e4e9e4","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d206ed9c5e22e16cf6e4e9e4: start","job_status":"start","queue_duration_s":2.216462,"scheduling_latency_s":2.216493}2223gitlab # [ 538.628694] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.704Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d206ed9c5e22e16cf6e4e9e4","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d206ed9c5e22e16cf6e4e9e4: done: 0.024694 sec","job_status":"done","queue_duration_s":2.216462,"scheduling_latency_s":2.216493,"redis_calls":13,"redis_duration_s":0.003451,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001028,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002423,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015836,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024694,"completed_at":"2026-10-02T14:18:15.704Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2224gitlab # [ 538.669068] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.706Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c26003e4bb8679238469174a","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c26003e4bb8679238469174a: start","job_status":"start","queue_duration_s":2.243521,"scheduling_latency_s":2.24355}2225gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/727998be4a72c681de6dc36cedfc3d5e9518c280/statuses, in 0.20 seconds)2226gitlab # [ 538.681941] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.731Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c26003e4bb8679238469174a","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c26003e4bb8679238469174a: done: 0.024105 sec","job_status":"done","queue_duration_s":2.243521,"scheduling_latency_s":2.24355,"redis_calls":13,"redis_duration_s":0.002961,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001002,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001959,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015932,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024105,"completed_at":"2026-10-02T14:18:15.730Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2227gitlab # [ 538.723262] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.732Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3c563467671bba27f995cabc","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3c563467671bba27f995cabc: start","job_status":"start","queue_duration_s":2.269487,"scheduling_latency_s":2.269521}2228gitlab # [ 538.736686] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:18:16 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/727998be4a72c681de6dc36cedfc3d5e9518c280/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1382229gitlab # [ 538.739658] gitaly[930]: time="2026-10-02T14:18:16.544Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNSG4WBSB4YQ4Z4CN9JNA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:18:46.542" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:16.542" grpc.time_ms=0.99 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2230gitlab # [ 538.753057] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.757Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3c563467671bba27f995cabc","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3c563467671bba27f995cabc: done: 0.024945 sec","job_status":"done","queue_duration_s":2.269487,"scheduling_latency_s":2.269521,"redis_calls":13,"redis_duration_s":0.003229,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001078,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002151,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016528,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024945,"completed_at":"2026-10-02T14:18:15.757Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2231gitlab # [ 538.793587] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.759Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1757aee43ff9849b1f09da81","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1757aee43ff9849b1f09da81: start","job_status":"start","queue_duration_s":2.296321,"scheduling_latency_s":2.296356}2232gitlab # [ 538.806540] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.783Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1757aee43ff9849b1f09da81","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1757aee43ff9849b1f09da81: done: 0.023867 sec","job_status":"done","queue_duration_s":2.296321,"scheduling_latency_s":2.296356,"redis_calls":13,"redis_duration_s":0.0029649999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000999,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001966,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015808,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023867,"completed_at":"2026-10-02T14:18:15.783Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2233gitlab # [ 538.847381] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.785Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"acef384fb55fc3a392f9b1df","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-acef384fb55fc3a392f9b1df: start","job_status":"start","queue_duration_s":2.322101,"scheduling_latency_s":2.322132}2234gitlab # [ 538.860372] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.809Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"acef384fb55fc3a392f9b1df","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-acef384fb55fc3a392f9b1df: done: 0.024399 sec","job_status":"done","queue_duration_s":2.322101,"scheduling_latency_s":2.322132,"redis_calls":13,"redis_duration_s":0.0033090000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001077,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002232,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015908,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024399,"completed_at":"2026-10-02T14:18:15.809Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2235gitlab # [ 538.901172] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.811Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9597d36c628a7cda76d3c56b","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9597d36c628a7cda76d3c56b: start","job_status":"start","queue_duration_s":2.348403,"scheduling_latency_s":2.348434}2236gitlab # [ 538.914014] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.835Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9597d36c628a7cda76d3c56b","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9597d36c628a7cda76d3c56b: done: 0.023813 sec","job_status":"done","queue_duration_s":2.348403,"scheduling_latency_s":2.348434,"redis_calls":13,"redis_duration_s":0.0028989999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001029,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00187,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015785,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023813,"completed_at":"2026-10-02T14:18:15.835Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2237gitlab # [ 538.955733] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.837Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e982833e11df1eab5ad1258","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e982833e11df1eab5ad1258: start","job_status":"start","queue_duration_s":2.37409,"scheduling_latency_s":2.374121}2238gitlab # [ 538.968758] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.862Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e982833e11df1eab5ad1258","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e982833e11df1eab5ad1258: done: 0.024739 sec","job_status":"done","queue_duration_s":2.37409,"scheduling_latency_s":2.374121,"redis_calls":13,"redis_duration_s":0.002911,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001017,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001894,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016172,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024739,"completed_at":"2026-10-02T14:18:15.862Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2239gitlab # [ 539.009101] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.864Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"13d1601ff4901ccad133fe4b","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-13d1601ff4901ccad133fe4b: start","job_status":"start","queue_duration_s":2.40076,"scheduling_latency_s":2.400791}2240gitlab # [ 539.021890] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.888Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"13d1601ff4901ccad133fe4b","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-13d1601ff4901ccad133fe4b: done: 0.024288 sec","job_status":"done","queue_duration_s":2.40076,"scheduling_latency_s":2.400791,"redis_calls":13,"redis_duration_s":0.002846,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00101,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001836,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016195,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024288,"completed_at":"2026-10-02T14:18:15.888Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2241gitlab # [ 539.062120] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.890Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fad08b691d1f4bd1f6baea78","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fad08b691d1f4bd1f6baea78: start","job_status":"start","queue_duration_s":2.427477,"scheduling_latency_s":2.427508}2242gitlab # [ 539.075091] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.915Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fad08b691d1f4bd1f6baea78","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fad08b691d1f4bd1f6baea78: done: 0.024826 sec","job_status":"done","queue_duration_s":2.427477,"scheduling_latency_s":2.427508,"redis_calls":13,"redis_duration_s":0.003016,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001074,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001942,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016178,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024826,"completed_at":"2026-10-02T14:18:15.915Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2243gitlab # [ 539.115861] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.917Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2ea32f56f27ade07665ddd6e","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2ea32f56f27ade07665ddd6e: start","job_status":"start","queue_duration_s":2.454257,"scheduling_latency_s":2.45429}2244gitlab # [ 539.131139] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.942Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2ea32f56f27ade07665ddd6e","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2ea32f56f27ade07665ddd6e: done: 0.024509 sec","job_status":"done","queue_duration_s":2.454257,"scheduling_latency_s":2.45429,"redis_calls":13,"redis_duration_s":0.002913,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001027,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001886,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016246,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024509,"completed_at":"2026-10-02T14:18:15.942Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2245gitlab # [ 539.172447] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.944Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b95b8c17b328ecf3832597c6","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b95b8c17b328ecf3832597c6: start","job_status":"start","queue_duration_s":2.481115,"scheduling_latency_s":2.481145}2246gitlab # [ 539.185364] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.967Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b95b8c17b328ecf3832597c6","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b95b8c17b328ecf3832597c6: done: 0.023456 sec","job_status":"done","queue_duration_s":2.481115,"scheduling_latency_s":2.481145,"redis_calls":13,"redis_duration_s":0.0029519999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001015,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001937,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015581,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023456,"completed_at":"2026-10-02T14:18:15.967Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2247gitlab # [ 539.226031] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.969Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0098cc635aa4f69ac4acd8b7","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0098cc635aa4f69ac4acd8b7: start","job_status":"start","queue_duration_s":2.506128,"scheduling_latency_s":2.506153}2248gitlab # [ 539.239056] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.985Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0098cc635aa4f69ac4acd8b7","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0098cc635aa4f69ac4acd8b7: done: 0.016118 sec","job_status":"done","queue_duration_s":2.506128,"scheduling_latency_s":2.506153,"redis_calls":13,"redis_duration_s":0.002052,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000709,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001343,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010661,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016118,"completed_at":"2026-10-02T14:18:15.985Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2249gitlab # [ 539.280142] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:15.987Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b8931bc076b55925914d0dd0","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b8931bc076b55925914d0dd0: start","job_status":"start","queue_duration_s":2.523679,"scheduling_latency_s":2.523696}2250gitlab # [ 539.292978] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.002Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b8931bc076b55925914d0dd0","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b8931bc076b55925914d0dd0: done: 0.015873 sec","job_status":"done","queue_duration_s":2.523679,"scheduling_latency_s":2.523696,"redis_calls":13,"redis_duration_s":0.002251,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000841,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00141,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01062,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015873,"completed_at":"2026-10-02T14:18:16.002Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2251gitlab # [ 539.334550] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.004Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"93bd0ca06c243ba7cbad9bfb","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-93bd0ca06c243ba7cbad9bfb: start","job_status":"start","queue_duration_s":2.54084,"scheduling_latency_s":2.540857}2252gitlab # [ 539.347588] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.020Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"93bd0ca06c243ba7cbad9bfb","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-93bd0ca06c243ba7cbad9bfb: done: 0.016527 sec","job_status":"done","queue_duration_s":2.54084,"scheduling_latency_s":2.540857,"redis_calls":13,"redis_duration_s":0.002221,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000757,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001464,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010738,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016527,"completed_at":"2026-10-02T14:18:16.020Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2253gitlab # [ 539.388716] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.022Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d052df1be388c9bc969e38d5","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d052df1be388c9bc969e38d5: start","job_status":"start","queue_duration_s":2.5587,"scheduling_latency_s":2.558718}2254gitlab # [ 539.401594] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.038Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d052df1be388c9bc969e38d5","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d052df1be388c9bc969e38d5: done: 0.016788 sec","job_status":"done","queue_duration_s":2.5587,"scheduling_latency_s":2.558718,"redis_calls":13,"redis_duration_s":0.002273,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000767,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001506,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011174,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016788,"completed_at":"2026-10-02T14:18:16.038Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2255gitlab # [ 539.442496] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.040Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3dc60b6ee9c0129082ba2e8e","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3dc60b6ee9c0129082ba2e8e: start","job_status":"start","queue_duration_s":2.577144,"scheduling_latency_s":2.577161}2256gitlab # [ 539.455549] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.063Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3dc60b6ee9c0129082ba2e8e","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3dc60b6ee9c0129082ba2e8e: done: 0.022775 sec","job_status":"done","queue_duration_s":2.577144,"scheduling_latency_s":2.577161,"redis_calls":13,"redis_duration_s":0.003044,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001221,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001823,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016446,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022775,"completed_at":"2026-10-02T14:18:16.063Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2257gitlab # [ 539.496463] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.065Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b4ee5039909f4fbdab99d6b8","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b4ee5039909f4fbdab99d6b8: start","job_status":"start","queue_duration_s":2.602015,"scheduling_latency_s":2.602039}2258gitlab # [ 539.509242] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.088Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b4ee5039909f4fbdab99d6b8","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b4ee5039909f4fbdab99d6b8: done: 0.022789 sec","job_status":"done","queue_duration_s":2.602015,"scheduling_latency_s":2.602039,"redis_calls":13,"redis_duration_s":0.002443,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000842,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001601,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016149,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022789,"completed_at":"2026-10-02T14:18:16.088Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2259gitlab # [ 539.550895] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.090Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"346456ced6dffcbbe827d1d5","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-346456ced6dffcbbe827d1d5: start","job_status":"start","queue_duration_s":2.626971,"scheduling_latency_s":2.626995}2260gitlab # [ 539.563929] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.111Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"346456ced6dffcbbe827d1d5","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-346456ced6dffcbbe827d1d5: done: 0.0214 sec","job_status":"done","queue_duration_s":2.626971,"scheduling_latency_s":2.626995,"redis_calls":13,"redis_duration_s":0.00254,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000867,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001673,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014866,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0214,"completed_at":"2026-10-02T14:18:16.111Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2261gitlab # [ 539.605048] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.113Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be46753c4ab1ac260e1a2afc","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be46753c4ab1ac260e1a2afc: start","job_status":"start","queue_duration_s":2.650246,"scheduling_latency_s":2.650266}2262gitlab # [ 539.617728] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.133Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be46753c4ab1ac260e1a2afc","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be46753c4ab1ac260e1a2afc: done: 0.019538 sec","job_status":"done","queue_duration_s":2.650246,"scheduling_latency_s":2.650266,"redis_calls":13,"redis_duration_s":0.002451,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000847,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001604,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01368,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019538,"completed_at":"2026-10-02T14:18:16.133Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2263gitlab # [ 539.659063] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.135Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"58198e2130769f6c81754064","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-58198e2130769f6c81754064: start","job_status":"start","queue_duration_s":2.671757,"scheduling_latency_s":2.671776}2264gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/727998be4a72c681de6dc36cedfc3d5e9518c280/statuses2265gitlab # [ 539.673196] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.336Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"58198e2130769f6c81754064","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-58198e2130769f6c81754064: done: 0.201398 sec","job_status":"done","queue_duration_s":2.671757,"scheduling_latency_s":2.671776,"redis_calls":13,"redis_duration_s":0.073638,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00158,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.072058,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024093,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.201398,"completed_at":"2026-10-02T14:18:16.336Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2266gitlab # [ 539.715366] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.338Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4d0b613c1fec11e4fab7c3de","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4d0b613c1fec11e4fab7c3de: start","job_status":"start","queue_duration_s":2.874974,"scheduling_latency_s":2.874998}2267gitlab # [ 539.729546] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.355Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4d0b613c1fec11e4fab7c3de","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4d0b613c1fec11e4fab7c3de: done: 0.017196 sec","job_status":"done","queue_duration_s":2.874974,"scheduling_latency_s":2.874998,"redis_calls":13,"redis_duration_s":0.002351,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000801,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00155,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011308,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017196,"completed_at":"2026-10-02T14:18:16.355Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2268gitlab # [ 539.774574] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.356Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"256fb0819d29e91dd4985992","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-256fb0819d29e91dd4985992: start","job_status":"start","queue_duration_s":2.893557,"scheduling_latency_s":2.893577}2269gitlab # [ 539.788242] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.372Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"256fb0819d29e91dd4985992","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-256fb0819d29e91dd4985992: done: 0.015854 sec","job_status":"done","queue_duration_s":2.893557,"scheduling_latency_s":2.893577,"redis_calls":13,"redis_duration_s":0.002228,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000783,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001445,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010421,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015854,"completed_at":"2026-10-02T14:18:16.372Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2270gitlab # [ 539.830144] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.374Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"58ba27ef683678991d137d87","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-58ba27ef683678991d137d87: start","job_status":"start","queue_duration_s":2.910782,"scheduling_latency_s":2.910801}2271gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/727998be4a72c681de6dc36cedfc3d5e9518c280/statuses, in 0.18 seconds)2272gitlab # [ 539.843563] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.390Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"58ba27ef683678991d137d87","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-58ba27ef683678991d137d87: done: 0.015852 sec","job_status":"done","queue_duration_s":2.910782,"scheduling_latency_s":2.910801,"redis_calls":13,"redis_duration_s":0.00219,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000732,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001458,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010431,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015852,"completed_at":"2026-10-02T14:18:16.389Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2273gitlab # [ 539.885795] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.391Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1fbbc42afdfd0b2344b266ed","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1fbbc42afdfd0b2344b266ed: start","job_status":"start","queue_duration_s":2.928044,"scheduling_latency_s":2.928063}2274gitlab # [ 539.899674] gitaly[930]: time="2026-10-02T14:18:17.731Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNTNN071MGQXS4X7CXFM5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:18:47.730" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:17.730" grpc.time_ms=0.951 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2275gitlab # [ 539.913981] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:18:17 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/727998be4a72c681de6dc36cedfc3d5e9518c280/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1152276gitlab # [ 539.916869] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.525Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1fbbc42afdfd0b2344b266ed","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1fbbc42afdfd0b2344b266ed: done: 0.133904 sec","job_status":"done","queue_duration_s":2.928044,"scheduling_latency_s":2.928063,"redis_calls":13,"redis_duration_s":0.00342,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001372,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002048,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.112392,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.133904,"completed_at":"2026-10-02T14:18:16.525Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2277gitlab # [ 539.958921] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.527Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7774596efc5fdb97dcb6a0cd","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7774596efc5fdb97dcb6a0cd: start","job_status":"start","queue_duration_s":3.064023,"scheduling_latency_s":3.064055}2278gitlab # [ 539.973289] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.548Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7774596efc5fdb97dcb6a0cd","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7774596efc5fdb97dcb6a0cd: done: 0.021074 sec","job_status":"done","queue_duration_s":3.064023,"scheduling_latency_s":3.064055,"redis_calls":13,"redis_duration_s":0.002565,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000895,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00167,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013963,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021074,"completed_at":"2026-10-02T14:18:16.548Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2279gitlab # [ 540.016062] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.551Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef20030b2beb6d64c8d327a9","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef20030b2beb6d64c8d327a9: start","job_status":"start","queue_duration_s":3.087759,"scheduling_latency_s":3.087787}2280gitlab # [ 540.030155] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.571Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef20030b2beb6d64c8d327a9","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef20030b2beb6d64c8d327a9: done: 0.020117 sec","job_status":"done","queue_duration_s":3.087759,"scheduling_latency_s":3.087787,"redis_calls":13,"redis_duration_s":0.002575,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000921,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001654,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013475,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020117,"completed_at":"2026-10-02T14:18:16.571Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2281gitlab # [ 540.071109] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.572Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"62dff3eaf7d2bebbbb22840b","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-62dff3eaf7d2bebbbb22840b: start","job_status":"start","queue_duration_s":3.109441,"scheduling_latency_s":3.109469}2282gitlab # [ 540.084158] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.593Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"62dff3eaf7d2bebbbb22840b","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-62dff3eaf7d2bebbbb22840b: done: 0.020256 sec","job_status":"done","queue_duration_s":3.109441,"scheduling_latency_s":3.109469,"redis_calls":13,"redis_duration_s":0.002394,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000896,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001498,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013102,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020256,"completed_at":"2026-10-02T14:18:16.592Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2283gitlab # [ 540.124883] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.595Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"220257103dc46faf60775787","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-220257103dc46faf60775787: start","job_status":"start","queue_duration_s":3.13162,"scheduling_latency_s":3.13165}2284gitlab # [ 540.138213] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.616Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"220257103dc46faf60775787","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-220257103dc46faf60775787: done: 0.021323 sec","job_status":"done","queue_duration_s":3.13162,"scheduling_latency_s":3.13165,"redis_calls":13,"redis_duration_s":0.0026579999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000848,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00181,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012707,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021323,"completed_at":"2026-10-02T14:18:16.616Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2285gitlab # [ 540.181666] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.618Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9c82df5bdded62f5f5954e9a","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9c82df5bdded62f5f5954e9a: start","job_status":"start","queue_duration_s":3.154823,"scheduling_latency_s":3.154849}2286gitlab # [ 540.195107] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.637Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9c82df5bdded62f5f5954e9a","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9c82df5bdded62f5f5954e9a: done: 0.019427 sec","job_status":"done","queue_duration_s":3.154823,"scheduling_latency_s":3.154849,"redis_calls":13,"redis_duration_s":0.0027429999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000998,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001745,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012793,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019427,"completed_at":"2026-10-02T14:18:16.637Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2287gitlab # [ 540.237684] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.639Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8c8f7af3f816e9608781003c","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8c8f7af3f816e9608781003c: start","job_status":"start","queue_duration_s":3.175663,"scheduling_latency_s":3.175685}2288gitlab # [ 540.251131] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.656Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8c8f7af3f816e9608781003c","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8c8f7af3f816e9608781003c: done: 0.01754 sec","job_status":"done","queue_duration_s":3.175663,"scheduling_latency_s":3.175685,"redis_calls":13,"redis_duration_s":0.002543,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000748,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001795,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010127,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01754,"completed_at":"2026-10-02T14:18:16.656Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2289gitlab # [ 540.292505] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.658Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4b3281c855a0476eb5f8a6e7","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4b3281c855a0476eb5f8a6e7: start","job_status":"start","queue_duration_s":3.194582,"scheduling_latency_s":3.194598}2290gitlab # [ 540.305343] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.674Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4b3281c855a0476eb5f8a6e7","created_at":"2026-10-02T14:18:13.332Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-19e9f825027e4c3f","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f0bf008b5a15a2cec245f41c85586469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:13.463Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4b3281c855a0476eb5f8a6e7: done: 0.016728 sec","job_status":"done","queue_duration_s":3.194582,"scheduling_latency_s":3.194598,"redis_calls":13,"redis_duration_s":0.002316,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000801,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001515,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011233,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016728,"completed_at":"2026-10-02T14:18:16.674Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2291gitlab # [ 540.346030] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.676Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"66dad4f5916c5301145f4d43","created_at":"2026-10-02T14:18:14.893Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-1a5ffea40f84442e","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"bb136775b0afba9d7ed0c3d9a57ee69e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:14.910Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-66dad4f5916c5301145f4d43: start","job_status":"start","queue_duration_s":1.766626,"scheduling_latency_s":1.766644}2292gitlab # [ 540.358920] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.696Z","message":"No existing merge request to be cleaned up."}2293gitlab # [ 540.360697] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.711Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"66dad4f5916c5301145f4d43","created_at":"2026-10-02T14:18:14.893Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-1a5ffea40f84442e","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"bb136775b0afba9d7ed0c3d9a57ee69e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:14.910Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-66dad4f5916c5301145f4d43: done: 0.034603 sec","job_status":"done","queue_duration_s":1.766626,"scheduling_latency_s":1.766644,"redis_calls":12,"redis_duration_s":0.002476,"redis_read_bytes":10,"redis_write_bytes":1323,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000436,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000859,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001181,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023388,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034603,"completed_at":"2026-10-02T14:18:16.711Z","load_balancing_strategy":"primary","db_duration_s":0.005167,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2294gitlab # [ 540.403075] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.712Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"1c263a3173b0ab52f2072faa","created_at":"2026-10-02T14:18:14.893Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-1a5ffea40f84442e","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"bb136775b0afba9d7ed0c3d9a57ee69e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:14.910Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-1c263a3173b0ab52f2072faa: start","job_status":"start","queue_duration_s":1.802605,"scheduling_latency_s":1.802623}2295gitlab # [ 540.416089] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.725Z","message":"No existing merge request to be cleaned up."}2296gitlab # [ 540.417809] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.736Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"1c263a3173b0ab52f2072faa","created_at":"2026-10-02T14:18:14.893Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-1a5ffea40f84442e","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"bb136775b0afba9d7ed0c3d9a57ee69e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:14.910Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-1c263a3173b0ab52f2072faa: done: 0.023255 sec","job_status":"done","queue_duration_s":1.802605,"scheduling_latency_s":1.802623,"redis_calls":10,"redis_duration_s":0.0020870000000000003,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000817,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00127,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015817,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023255,"completed_at":"2026-10-02T14:18:16.736Z","load_balancing_strategy":"primary","db_duration_s":0.002231,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2297gitlab # [ 540.457975] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.737Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"bc5f628b0ee53cbcbae4a92b","created_at":"2026-10-02T14:18:14.893Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-1a5ffea40f84442e","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"bb136775b0afba9d7ed0c3d9a57ee69e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:14.910Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-bc5f628b0ee53cbcbae4a92b: start","job_status":"start","queue_duration_s":1.827142,"scheduling_latency_s":1.827158}2298gitlab # [ 540.470709] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.750Z","message":"No existing merge request to be cleaned up."}2299gitlab # [ 540.472449] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.759Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"bc5f628b0ee53cbcbae4a92b","created_at":"2026-10-02T14:18:14.893Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-1a5ffea40f84442e","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"bb136775b0afba9d7ed0c3d9a57ee69e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:14.910Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-bc5f628b0ee53cbcbae4a92b: done: 0.02188 sec","job_status":"done","queue_duration_s":1.827142,"scheduling_latency_s":1.827158,"redis_calls":10,"redis_duration_s":0.0020689999999999997,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000815,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001254,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015446,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02188,"completed_at":"2026-10-02T14:18:16.759Z","load_balancing_strategy":"primary","db_duration_s":0.002076,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2300gitlab # [ 540.512517] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.761Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a47367e4cabdfcc087bdaaea","created_at":"2026-10-02T14:18:14.893Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-1a5ffea40f84442e","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"bb136775b0afba9d7ed0c3d9a57ee69e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:14.910Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a47367e4cabdfcc087bdaaea: start","job_status":"start","queue_duration_s":1.851155,"scheduling_latency_s":1.851172}2301gitlab # [ 540.525199] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.773Z","message":"No existing merge request to be cleaned up."}2302gitlab # [ 540.526932] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:16.783Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a47367e4cabdfcc087bdaaea","created_at":"2026-10-02T14:18:14.893Z","trace_propagation_headers":{"sentry-trace":"b8384ebe5a894da18ecf382aa26b5046-1a5ffea40f84442e","baggage":"sentry-trace_id=b8384ebe5a894da18ecf382aa26b5046,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"bb136775b0afba9d7ed0c3d9a57ee69e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:18:14.910Z","job_size_bytes":2,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a47367e4cabdfcc087bdaaea: done: 0.021935 sec","job_status":"done","queue_duration_s":1.851155,"scheduling_latency_s":1.851172,"redis_calls":10,"redis_duration_s":0.00194,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000822,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001118,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015439,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021935,"completed_at":"2026-10-02T14:18:16.783Z","load_balancing_strategy":"primary","db_duration_s":0.002016,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2303gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/727998be4a72c681de6dc36cedfc3d5e9518c280/statuses2304gitlab # [ 541.009886] gitaly[930]: time="2026-10-02T14:18:18.919Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNVT879535K1ARZMV1A2Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:18:48.017" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:18.918" grpc.time_ms=0.99 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2305gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/727998be4a72c681de6dc36cedfc3d5e9518c280/statuses, in 0.19 seconds)2306gitlab # [ 541.058108] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:18:18 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/727998be4a72c681de6dc36cedfc3d5e9518c280/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1422307gitlab # [ 541.566371] nix-daemon[4288]: accepted connection from pid 5500, user nixbot2308gitlab # [ 541.705743] gitaly[930]: time="2026-10-02T14:18:19.615Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNWFSNK353Z56T8JXM6PK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:18:49.614" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:19.614" grpc.time_ms=0.918 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2309gitlab # [ 541.779581] gitaly[930]: time="2026-10-02T14:18:19.689Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFNWFSNK353Z56T8JXM6PK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T14:18:29.266" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:18:19.677" grpc.time_ms=12.22 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2310gitlab # [ 541.833521] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:19.741Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"481e3d6641ee7f0165aaaca6","created_at":"2026-10-02T14:18:19.733Z","trace_propagation_headers":{"sentry-trace":"ca670d031b5f4b80ba2442c85333f84b-0e9900bb2b434ae2","baggage":"sentry-trace_id=ca670d031b5f4b80ba2442c85333f84b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFNWFSNK353Z56T8JXM6PK","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:19.739Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-481e3d6641ee7f0165aaaca6: start","job_status":"start","queue_duration_s":0.001731,"scheduling_latency_s":0.001775}2311gitlab # [ 541.860060] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:18:19 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/727998be4a72c681de6dc36cedfc3d5e9518c280 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 2492312gitlab # [ 541.864955] nixbot[4119]: {"ts": "2026-10-02T14:18:19.765Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/727998be4a72c681de6dc36cedfc3d5e9518c280 \"HTTP/1.1 201 Created\""}2313gitlab # [ 541.941215] gitaly[930]: time="2026-10-02T14:18:19.851Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNWQVDH8BT5PV3TJMV3SX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:18:49.849" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:19.850" grpc.time_ms=0.985 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2314gitlab # [ 542.003678] gitaly[930]: time="2026-10-02T14:18:19.913Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFNWQVDH8BT5PV3TJMV3SX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T14:18:29.010" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:18:19.901" grpc.time_ms=12.533 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2315gitlab # [ 542.028380] gitaly[930]: time="2026-10-02T14:18:19.932Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFNWFSNK353Z56T8JXM6PK grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:18:19.932" grpc.time_ms=0.596 pid=930 span.kind=server system=grpc2316gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/727998be4a72c681de6dc36cedfc3d5e9518c280/statuses2317gitlab # [ 542.055899] gitaly[930]: time="2026-10-02T14:18:19.966Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFNWFSNK353Z56T8JXM6PK grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:18:19.965" grpc.time_ms=0.655 pid=930 span.kind=server system=grpc2318gitlab # [ 542.075032] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:18:19 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/727998be4a72c681de6dc36cedfc3d5e9518c280 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 2122319gitlab # [ 542.079728] nixbot[4119]: {"ts": "2026-10-02T14:18:19.985Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/727998be4a72c681de6dc36cedfc3d5e9518c280 \"HTTP/1.1 201 Created\""}2320gitlab # [ 542.102996] gitaly[930]: time="2026-10-02T14:18:20.013Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFNWFSNK353Z56T8JXM6PK grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:18:20.011" grpc.time_ms=0.568 pid=930 span.kind=server system=grpc2321gitlab # [ 542.119059] gitaly[930]: time="2026-10-02T14:18:20.017Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNWFSNK353Z56T8JXM6PK forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-02T14:18:49.693" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:18:19.894" operations="[repack_refs]" pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2322gitlab # [ 542.132696] gitaly[930]: time="2026-10-02T14:18:20.020Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNWFSNK353Z56T8JXM6PK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-02T14:18:49.693" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:18:19.894" pid=930 remote_ip="2001:db8:1::1" repository_info="{false {6 703 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-10-02 14:16:00.31320836 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root2323gitlab # [ 542.147408] gitaly[930]: time="2026-10-02T14:18:20.024Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNWFSNK353Z56T8JXM6PK diskcache=33ac949b-0971-4d36-99ea-5673abbcca40 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-02T14:18:49.693" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:18:19.894" pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2324gitlab # [ 542.158697] gitaly[930]: time="2026-10-02T14:18:20.025Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=104 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7142 command.oublock=0 command.real_time_ms=120 command.spawn_token_fork_ms=1 command.system_time_ms=61 command.user_time_ms=43 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNWFSNK353Z56T8JXM6PK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-02T14:18:49.693" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:18:19.894" grpc.time_ms=130.611 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2325gitlab # [ 542.173908] gitaly[930]: time="2026-10-02T14:18:20.061Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNWFSNK353Z56T8JXM6PK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-02T14:18:49.693" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:18:19.894" pid=930 remote_ip="2001:db8:1::1" span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root2326gitlab # [ 542.188292] gitaly[930]: time="2026-10-02T14:18:20.061Z" level=info msg="optimized repository" clean_stale_data_time=40.31846ms clean_work_trees_time="135.4µs" commit_graph_time="1.02µs" correlation_id=01M3YFNWFSNK353Z56T8JXM6PK gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="1.7µs" pid=930 prune_time=160ns relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="2.2µs" storage_name=default system=housekeeping total_time=40.51048ms2327gitlab # [ 542.194649] gitaly[930]: time="2026-10-02T14:18:20.061Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNWFSNK353Z56T8JXM6PK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-02T14:18:49.693" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:18:19.894" pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2328gitlab # [ 542.204784] gitaly[930]: time="2026-10-02T14:18:20.095Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNWZEYCNR7SRYYB5DYM26 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:18:50.092" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:20.092" grpc.time_ms=2.297 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2329gitlab # [ 542.224073] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:18:20 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/727998be4a72c681de6dc36cedfc3d5e9518c280/statuses HTTP/1.1" 200 1313 "" "curl/8.21.0" 1192330gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/727998be4a72c681de6dc36cedfc3d5e9518c280/statuses, in 0.17 seconds)2331gitlab # [ 542.228375] gitaly[930]: time="2026-10-02T14:18:20.138Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNWZ3DR2G6KP9VMS6KE8G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:18:50.132" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:20.132" grpc.time_ms=0.808 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2332gitlab # [ 542.255426] gitaly[930]: time="2026-10-02T14:18:20.165Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFNWFSNK353Z56T8JXM6PK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-02T14:18:50.064" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:20.164" grpc.time_ms=0.904 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2333gitlab # [ 542.285144] gitaly[930]: time="2026-10-02T14:18:20.195Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M3YFNWZ3DR2G6KP9VMS6KE8G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T14:18:30.175" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:18:20.185" grpc.time_ms=9.743 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2334gitlab # [ 542.308191] gitaly[930]: time="2026-10-02T14:18:20.218Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNWFSNK353Z56T8JXM6PK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:18:50.015" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:18:20.215" grpc.time_ms=2.158 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2335gitlab # [ 542.327232] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:20.235Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"a3856c3142738d1e8e837d18","created_at":"2026-10-02T14:18:20.232Z","trace_propagation_headers":{"sentry-trace":"ca670d031b5f4b80ba2442c85333f84b-5b54ff55ba324cda","baggage":"sentry-trace_id=ca670d031b5f4b80ba2442c85333f84b,sentry-release=Unknown"},"correlation_id":"01M3YFNWFSNK353Z56T8JXM6PK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","duplicate-of":"481e3d6641ee7f0165aaaca6","job_size_bytes":3,"pid":3989,"job_status":"deduplicated","message":"PipelineProcessWorker JID-a3856c3142738d1e8e837d18: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}2336gitlab # [ 542.353524] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:20.243Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"481e3d6641ee7f0165aaaca6","created_at":"2026-10-02T14:18:19.733Z","trace_propagation_headers":{"sentry-trace":"ca670d031b5f4b80ba2442c85333f84b-0e9900bb2b434ae2","baggage":"sentry-trace_id=ca670d031b5f4b80ba2442c85333f84b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFNWFSNK353Z56T8JXM6PK","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:19.739Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":3989,"job_status":"rescheduled","message":"PipelineProcessWorker JID-481e3d6641ee7f0165aaaca6: rescheduled"}2337gitlab # [ 542.375611] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:20.245Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"f853b0f69a4d8722b75e1a12","created_at":"2026-10-02T14:18:20.243Z","trace_propagation_headers":{"sentry-trace":"ca670d031b5f4b80ba2442c85333f84b-5b54ff55ba324cda","baggage":"sentry-trace_id=ca670d031b5f4b80ba2442c85333f84b,sentry-release=Unknown"},"correlation_id":"01M3YFNWFSNK353Z56T8JXM6PK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","duplicate-of":"e7b328c620aff3445345dcd4","job_size_bytes":3,"pid":3989,"job_status":"deduplicated","message":"PipelineProcessWorker JID-f853b0f69a4d8722b75e1a12: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}2338gitlab # [ 542.396905] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:20.251Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"481e3d6641ee7f0165aaaca6","created_at":"2026-10-02T14:18:19.733Z","trace_propagation_headers":{"sentry-trace":"ca670d031b5f4b80ba2442c85333f84b-0e9900bb2b434ae2","baggage":"sentry-trace_id=ca670d031b5f4b80ba2442c85333f84b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFNWFSNK353Z56T8JXM6PK","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:19.739Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-481e3d6641ee7f0165aaaca6: done: 0.509942 sec","job_status":"done","queue_duration_s":0.001731,"scheduling_latency_s":0.001775,"gitaly_calls":3,"gitaly_duration_s":0.156385,"redis_calls":78,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.03144,"redis_read_bytes":312,"redis_write_bytes":20819,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.00145,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":11,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001683,"redis_cache_read_bytes":20,"redis_cache_write_bytes":948,"redis_queues_calls":23,"redis_queues_duration_s":0.006675,"redis_queues_read_bytes":25,"redis_queues_write_bytes":13381,"redis_queues_metadata_calls":24,"redis_queues_metadata_duration_s":0.013747,"redis_queues_metadata_read_bytes":188,"redis_queues_metadata_write_bytes":3877,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000313,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00413,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001992,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.075,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":16,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.011,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.014,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.047,"db_ci_duration_s":0.028,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.257183,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.42765912,"duration_s":0.509942,"completed_at":"2026-10-02T14:18:20.251Z","load_balancing_strategy":"primary","db_duration_s":0.077476,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2339gitlab # [ 542.456452] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:20.255Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"34c5722e393b802b2d4559fe","created_at":"2026-10-02T14:18:19.880Z","trace_propagation_headers":{"sentry-trace":"ca670d031b5f4b80ba2442c85333f84b-5b54ff55ba324cda","baggage":"sentry-trace_id=ca670d031b5f4b80ba2442c85333f84b,sentry-release=Unknown"},"correlation_id":"01M3YFNWFSNK353Z56T8JXM6PK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:19.883Z","job_size_bytes":23,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-34c5722e393b802b2d4559fe: start","job_status":"start","queue_duration_s":0.371173,"scheduling_latency_s":0.371201}2340gitlab # [ 542.474246] nixbot[4119]: {"ts": "2026-10-02T14:18:20.265Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/727998be4a72c681de6dc36cedfc3d5e9518c280 \"HTTP/1.1 201 Created\""}2341gitlab # [ 542.477309] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:18:20 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/727998be4a72c681de6dc36cedfc3d5e9518c280 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 2602342gitlab # [ 542.480094] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:20.324Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"34c5722e393b802b2d4559fe","created_at":"2026-10-02T14:18:19.880Z","trace_propagation_headers":{"sentry-trace":"ca670d031b5f4b80ba2442c85333f84b-5b54ff55ba324cda","baggage":"sentry-trace_id=ca670d031b5f4b80ba2442c85333f84b,sentry-release=Unknown"},"correlation_id":"01M3YFNWFSNK353Z56T8JXM6PK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:19.883Z","job_size_bytes":23,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-34c5722e393b802b2d4559fe: done: 0.069079 sec","job_status":"done","queue_duration_s":0.371173,"scheduling_latency_s":0.371201,"redis_calls":7,"redis_duration_s":0.0018319999999999999,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001326,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000506,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.047157,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.069079,"completed_at":"2026-10-02T14:18:20.324Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017336,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2343gitlab # [ 542.526015] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:20.326Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"72996e0aff79e6c33e15c755","created_at":"2026-10-02T14:18:20.048Z","trace_propagation_headers":{"sentry-trace":"ca670d031b5f4b80ba2442c85333f84b-5b54ff55ba324cda","baggage":"sentry-trace_id=ca670d031b5f4b80ba2442c85333f84b,sentry-release=Unknown"},"correlation_id":"01M3YFNWFSNK353Z56T8JXM6PK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:20.052Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-72996e0aff79e6c33e15c755: start","job_status":"start","queue_duration_s":0.274143,"scheduling_latency_s":0.274237}2344gitlab # [ 542.544114] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:20.370Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"72996e0aff79e6c33e15c755","created_at":"2026-10-02T14:18:20.048Z","trace_propagation_headers":{"sentry-trace":"ca670d031b5f4b80ba2442c85333f84b-5b54ff55ba324cda","baggage":"sentry-trace_id=ca670d031b5f4b80ba2442c85333f84b,sentry-release=Unknown"},"correlation_id":"01M3YFNWFSNK353Z56T8JXM6PK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:20.052Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-72996e0aff79e6c33e15c755: done: 0.044612 sec","job_status":"done","queue_duration_s":0.274143,"scheduling_latency_s":0.274237,"redis_calls":7,"redis_duration_s":0.0016899999999999999,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001207,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000483,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031676,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.044612,"completed_at":"2026-10-02T14:18:20.370Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005907,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2345gitlab # [ 542.589902] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:20.372Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"dbe96e96e6fd8db7a081dcf1","created_at":"2026-10-02T14:18:20.069Z","trace_propagation_headers":{"sentry-trace":"ca670d031b5f4b80ba2442c85333f84b-5b54ff55ba324cda","baggage":"sentry-trace_id=ca670d031b5f4b80ba2442c85333f84b,sentry-release=Unknown"},"correlation_id":"01M3YFNWFSNK353Z56T8JXM6PK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:20.076Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-dbe96e96e6fd8db7a081dcf1: start","job_status":"start","queue_duration_s":0.296795,"scheduling_latency_s":0.296825}2346gitlab # [ 542.606527] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:20.424Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"dbe96e96e6fd8db7a081dcf1","created_at":"2026-10-02T14:18:20.069Z","trace_propagation_headers":{"sentry-trace":"ca670d031b5f4b80ba2442c85333f84b-5b54ff55ba324cda","baggage":"sentry-trace_id=ca670d031b5f4b80ba2442c85333f84b,sentry-release=Unknown"},"correlation_id":"01M3YFNWFSNK353Z56T8JXM6PK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:20.076Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-dbe96e96e6fd8db7a081dcf1: done: 0.051949 sec","job_status":"done","queue_duration_s":0.296795,"scheduling_latency_s":0.296825,"redis_calls":7,"redis_duration_s":0.001968,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001354,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000614,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039022,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.051949,"completed_at":"2026-10-02T14:18:20.424Z","load_balancing_strategy":"primary","db_duration_s":0.006737,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2347gitlab # [ 542.651337] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:20.427Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"7ec33d3551f858c11c368890","created_at":"2026-10-02T14:18:20.097Z","trace_propagation_headers":{"sentry-trace":"ca670d031b5f4b80ba2442c85333f84b-5b54ff55ba324cda","baggage":"sentry-trace_id=ca670d031b5f4b80ba2442c85333f84b,sentry-release=Unknown"},"correlation_id":"01M3YFNWFSNK353Z56T8JXM6PK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:20.101Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-7ec33d3551f858c11c368890: start","job_status":"start","queue_duration_s":0.325998,"scheduling_latency_s":0.32603}2348gitlab # [ 542.667975] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:20.483Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"7ec33d3551f858c11c368890","created_at":"2026-10-02T14:18:20.097Z","trace_propagation_headers":{"sentry-trace":"ca670d031b5f4b80ba2442c85333f84b-5b54ff55ba324cda","baggage":"sentry-trace_id=ca670d031b5f4b80ba2442c85333f84b,sentry-release=Unknown"},"correlation_id":"01M3YFNWFSNK353Z56T8JXM6PK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:20.101Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-7ec33d3551f858c11c368890: done: 0.055717 sec","job_status":"done","queue_duration_s":0.325998,"scheduling_latency_s":0.32603,"redis_calls":10,"redis_duration_s":0.002581,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001334,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000742,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000505,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041127,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.055717,"completed_at":"2026-10-02T14:18:20.483Z","load_balancing_strategy":"primary","db_duration_s":0.008294,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2349gitlab # [ 542.713291] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:20.486Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"332dfd2ecaedec924bac6d6c","created_at":"2026-10-02T14:18:20.102Z","trace_propagation_headers":{"sentry-trace":"ca670d031b5f4b80ba2442c85333f84b-5b54ff55ba324cda","baggage":"sentry-trace_id=ca670d031b5f4b80ba2442c85333f84b,sentry-release=Unknown"},"correlation_id":"01M3YFNWFSNK353Z56T8JXM6PK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:20.106Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-332dfd2ecaedec924bac6d6c: start","job_status":"start","queue_duration_s":0.379277,"scheduling_latency_s":0.379308}2350gitlab # [ 542.730598] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:20.535Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"332dfd2ecaedec924bac6d6c","created_at":"2026-10-02T14:18:20.102Z","trace_propagation_headers":{"sentry-trace":"ca670d031b5f4b80ba2442c85333f84b-5b54ff55ba324cda","baggage":"sentry-trace_id=ca670d031b5f4b80ba2442c85333f84b,sentry-release=Unknown"},"correlation_id":"01M3YFNWFSNK353Z56T8JXM6PK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:20.106Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-332dfd2ecaedec924bac6d6c: done: 0.049239 sec","job_status":"done","queue_duration_s":0.379277,"scheduling_latency_s":0.379308,"redis_calls":7,"redis_duration_s":0.001791,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001301,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00049,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037616,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.049239,"completed_at":"2026-10-02T14:18:20.535Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006371,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2351gitlab # [ 542.776167] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:20.536Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"d8f710d97195de20033b3500","created_at":"2026-10-02T14:18:20.107Z","trace_propagation_headers":{"sentry-trace":"ca670d031b5f4b80ba2442c85333f84b-5b54ff55ba324cda","baggage":"sentry-trace_id=ca670d031b5f4b80ba2442c85333f84b,sentry-release=Unknown"},"correlation_id":"01M3YFNWFSNK353Z56T8JXM6PK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:20.112Z","job_size_bytes":19,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-d8f710d97195de20033b3500: start","job_status":"start","queue_duration_s":0.424464,"scheduling_latency_s":0.424488}2352gitlab # [ 542.793077] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:20.566Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"d8f710d97195de20033b3500","created_at":"2026-10-02T14:18:20.107Z","trace_propagation_headers":{"sentry-trace":"ca670d031b5f4b80ba2442c85333f84b-5b54ff55ba324cda","baggage":"sentry-trace_id=ca670d031b5f4b80ba2442c85333f84b,sentry-release=Unknown"},"correlation_id":"01M3YFNWFSNK353Z56T8JXM6PK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:20.112Z","job_size_bytes":19,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-d8f710d97195de20033b3500: done: 0.030137 sec","job_status":"done","queue_duration_s":0.424464,"scheduling_latency_s":0.424488,"redis_calls":8,"redis_duration_s":0.001442,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001109,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000333,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020879,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030137,"completed_at":"2026-10-02T14:18:20.566Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004566,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2353gitlab # [ 542.837559] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:20.568Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"5a80952eca20e66ddfb6b79f","created_at":"2026-10-02T14:18:20.113Z","trace_propagation_headers":{"sentry-trace":"ca670d031b5f4b80ba2442c85333f84b-5b54ff55ba324cda","baggage":"sentry-trace_id=ca670d031b5f4b80ba2442c85333f84b,sentry-release=Unknown"},"correlation_id":"01M3YFNWFSNK353Z56T8JXM6PK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:20.117Z","job_size_bytes":66,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-5a80952eca20e66ddfb6b79f: start","job_status":"start","queue_duration_s":0.450399,"scheduling_latency_s":0.450415}2354gitlab # [ 542.857452] gitaly[930]: time="2026-10-02T14:18:20.705Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFNXHYJXN94CG1H069NQAK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:18:20.695" grpc.time_ms=10.419 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc2355gitlab # [ 542.871745] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:18:20 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1002356gitlab # [ 542.874177] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:20.602Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"5a80952eca20e66ddfb6b79f","created_at":"2026-10-02T14:18:20.113Z","trace_propagation_headers":{"sentry-trace":"ca670d031b5f4b80ba2442c85333f84b-5b54ff55ba324cda","baggage":"sentry-trace_id=ca670d031b5f4b80ba2442c85333f84b,sentry-release=Unknown"},"correlation_id":"01M3YFNWFSNK353Z56T8JXM6PK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:20.117Z","job_size_bytes":66,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-5a80952eca20e66ddfb6b79f: done: 0.034346 sec","job_status":"done","queue_duration_s":0.450399,"scheduling_latency_s":0.450415,"redis_calls":7,"redis_duration_s":0.005721,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00129,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004431,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021951,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034346,"completed_at":"2026-10-02T14:18:20.602Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003351,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2357gitlab # [ 542.921292] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:20.604Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"bcb85976c387f53d5fff7458","created_at":"2026-10-02T14:18:20.119Z","trace_propagation_headers":{"sentry-trace":"ca670d031b5f4b80ba2442c85333f84b-5b54ff55ba324cda","baggage":"sentry-trace_id=ca670d031b5f4b80ba2442c85333f84b,sentry-release=Unknown"},"correlation_id":"01M3YFNWFSNK353Z56T8JXM6PK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:20.122Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-bcb85976c387f53d5fff7458: start","job_status":"start","queue_duration_s":0.481019,"scheduling_latency_s":0.481037}2358gitlab # [ 542.939208] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:20.668Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"bcb85976c387f53d5fff7458","created_at":"2026-10-02T14:18:20.119Z","trace_propagation_headers":{"sentry-trace":"ca670d031b5f4b80ba2442c85333f84b-5b54ff55ba324cda","baggage":"sentry-trace_id=ca670d031b5f4b80ba2442c85333f84b,sentry-release=Unknown"},"correlation_id":"01M3YFNWFSNK353Z56T8JXM6PK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:20.122Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-bcb85976c387f53d5fff7458: done: 0.064848 sec","job_status":"done","queue_duration_s":0.481019,"scheduling_latency_s":0.481037,"redis_calls":7,"redis_duration_s":0.003162,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002513,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000649,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043799,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.064848,"completed_at":"2026-10-02T14:18:20.668Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009594,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2359gitlab # [ 542.985331] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:20.671Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"db6dbcf1d8df297545a94fd6","created_at":"2026-10-02T14:18:20.132Z","trace_propagation_headers":{"sentry-trace":"ca670d031b5f4b80ba2442c85333f84b-5b54ff55ba324cda","baggage":"sentry-trace_id=ca670d031b5f4b80ba2442c85333f84b,sentry-release=Unknown"},"correlation_id":"01M3YFNWFSNK353Z56T8JXM6PK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:20.136Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-db6dbcf1d8df297545a94fd6: start","job_status":"start","queue_duration_s":0.53499,"scheduling_latency_s":0.535015}2360gitlab # [ 543.003122] gitlab-workhorse[816]: time="2026-10-02T14:18:20Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672361gitlab # [ 543.006475] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:18:20 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 952362gitlab # [ 543.011233] gitaly[930]: time="2026-10-02T14:18:20.803Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNXN578X5P0345HBGTDS9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:18:20.789" pid=930 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2363gitlab # [ 543.022687] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:20.708Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"db6dbcf1d8df297545a94fd6","created_at":"2026-10-02T14:18:20.132Z","trace_propagation_headers":{"sentry-trace":"ca670d031b5f4b80ba2442c85333f84b-5b54ff55ba324cda","baggage":"sentry-trace_id=ca670d031b5f4b80ba2442c85333f84b,sentry-release=Unknown"},"correlation_id":"01M3YFNWFSNK353Z56T8JXM6PK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:20.136Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-db6dbcf1d8df297545a94fd6: done: 0.037009 sec","job_status":"done","queue_duration_s":0.53499,"scheduling_latency_s":0.535015,"redis_calls":7,"redis_duration_s":0.001946,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001387,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000559,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022004,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.037009,"completed_at":"2026-10-02T14:18:20.708Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007795,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2364gitlab # [ 543.069888] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:20.710Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"58d6360d82ffb9063c8ec78e","created_at":"2026-10-02T14:18:20.136Z","trace_propagation_headers":{"sentry-trace":"ca670d031b5f4b80ba2442c85333f84b-5b54ff55ba324cda","baggage":"sentry-trace_id=ca670d031b5f4b80ba2442c85333f84b,sentry-release=Unknown"},"correlation_id":"01M3YFNWFSNK353Z56T8JXM6PK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:20.139Z","job_size_bytes":6,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-58d6360d82ffb9063c8ec78e: start","job_status":"start","queue_duration_s":0.570963,"scheduling_latency_s":0.570984}2365gitlab # [ 543.088006] gitaly[930]: time="2026-10-02T14:18:20.803Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNXN578X5P0345HBGTDS9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:18:20.789" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc2366gitlab # [ 543.102919] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:18:20 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 832367gitlab # [ 543.105316] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:20.729Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"58d6360d82ffb9063c8ec78e","created_at":"2026-10-02T14:18:20.136Z","trace_propagation_headers":{"sentry-trace":"ca670d031b5f4b80ba2442c85333f84b-5b54ff55ba324cda","baggage":"sentry-trace_id=ca670d031b5f4b80ba2442c85333f84b,sentry-release=Unknown"},"correlation_id":"01M3YFNWFSNK353Z56T8JXM6PK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:20.139Z","job_size_bytes":6,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-58d6360d82ffb9063c8ec78e: done: 0.018472 sec","job_status":"done","queue_duration_s":0.570963,"scheduling_latency_s":0.570984,"redis_calls":7,"redis_duration_s":0.001898,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000912,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000986,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010791,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018472,"completed_at":"2026-10-02T14:18:20.729Z","load_balancing_strategy":"primary","db_duration_s":0.000529,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2368gitlab # [ 543.150836] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:20.730Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"3a143d9efcd5d8c77327663e","created_at":"2026-10-02T14:18:20.143Z","trace_propagation_headers":{"sentry-trace":"ca670d031b5f4b80ba2442c85333f84b-5b54ff55ba324cda","baggage":"sentry-trace_id=ca670d031b5f4b80ba2442c85333f84b,sentry-release=Unknown"},"correlation_id":"01M3YFNWFSNK353Z56T8JXM6PK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:20.146Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-3a143d9efcd5d8c77327663e: start","job_status":"start","queue_duration_s":0.58424,"scheduling_latency_s":0.584258}2369gitlab # [ 543.168416] gitaly[930]: time="2026-10-02T14:18:20.804Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFNXN578X5P0345HBGTDS9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:18:20.789" grpc.time_ms=14.904 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc2370gitlab # [ 543.184145] gitlab-workhorse[816]: time="2026-10-02T14:18:21Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672371gitlab # [ 543.186141] gitlab-workhorse[816]: gitlab 2001:db8:1::1 - - [2026/10/02:14:18:21 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1062372gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/727998be4a72c681de6dc36cedfc3d5e9518c280/statuses2373gitlab # [ 543.188219] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:20.759Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"3a143d9efcd5d8c77327663e","created_at":"2026-10-02T14:18:20.143Z","trace_propagation_headers":{"sentry-trace":"ca670d031b5f4b80ba2442c85333f84b-5b54ff55ba324cda","baggage":"sentry-trace_id=ca670d031b5f4b80ba2442c85333f84b,sentry-release=Unknown"},"correlation_id":"01M3YFNWFSNK353Z56T8JXM6PK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:20.146Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-3a143d9efcd5d8c77327663e: done: 0.028716 sec","job_status":"done","queue_duration_s":0.58424,"scheduling_latency_s":0.584258,"redis_calls":7,"redis_duration_s":0.0011740000000000001,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000861,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000313,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019443,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028716,"completed_at":"2026-10-02T14:18:20.759Z","load_balancing_strategy":"primary","db_duration_s":0.005208,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2374gitlab # [ 543.232397] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:20.761Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"e7b328c620aff3445345dcd4","created_at":"2026-10-02T14:18:20.239Z","trace_propagation_headers":{"sentry-trace":"f160687c78284162b43077a6c064ed17-792a162d921742b2","baggage":"sentry-trace_id=f160687c78284162b43077a6c064ed17,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFNWZ3DR2G6KP9VMS6KE8G","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:20.244Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-e7b328c620aff3445345dcd4: start","job_status":"start","queue_duration_s":0.516177,"scheduling_latency_s":0.516195}2375gitlab # [ 543.248059] gitaly[930]: time="2026-10-02T14:18:20.913Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFNWZ3DR2G6KP9VMS6KE8G grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:18:20.912" grpc.time_ms=0.541 pid=930 span.kind=server system=grpc2376gitlab # [ 543.258354] gitaly[930]: time="2026-10-02T14:18:20.934Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFNXSK2VZYFCEVZTDFV565 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:18:20.923" grpc.time_ms=10.189 pid=930 remote_ip="2001:db8:1::1" span.kind=server system=grpc2377gitlab # [ 543.274116] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:21.139Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"e7b328c620aff3445345dcd4","created_at":"2026-10-02T14:18:20.239Z","trace_propagation_headers":{"sentry-trace":"f160687c78284162b43077a6c064ed17-792a162d921742b2","baggage":"sentry-trace_id=f160687c78284162b43077a6c064ed17,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFNWZ3DR2G6KP9VMS6KE8G","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:20.244Z","job_size_bytes":3,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-e7b328c620aff3445345dcd4: done: 0.378537 sec","job_status":"done","queue_duration_s":0.516177,"scheduling_latency_s":0.516195,"gitaly_calls":3,"gitaly_duration_s":0.14353,"redis_calls":65,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.025203,"redis_read_bytes":144,"redis_write_bytes":16871,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000748,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":11,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.004041,"redis_cache_read_bytes":20,"redis_cache_write_bytes":948,"redis_queues_calls":20,"redis_queues_duration_s":0.004297,"redis_queues_read_bytes":21,"redis_queues_write_bytes":11220,"redis_queues_metadata_calls":18,"redis_queues_metadata_duration_s":0.007376,"redis_queues_metadata_read_bytes":28,"redis_queues_metadata_write_bytes":2886,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.001787,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.002482,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.003724,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.049,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.008,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.008,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.029,"db_ci_duration_s":0.02,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.172445,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.33690448,"duration_s":0.378537,"completed_at":"2026-10-02T14:18:21.139Z","load_balancing_strategy":"primary","db_duration_s":0.050959,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2378gitlab # [ 543.329853] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:21.141Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"8bdc4686617c1caf6c1891a6","created_at":"2026-10-02T14:18:20.847Z","trace_propagation_headers":{"sentry-trace":"f160687c78284162b43077a6c064ed17-0c7b56dfa619473b","baggage":"sentry-trace_id=f160687c78284162b43077a6c064ed17,sentry-release=Unknown"},"correlation_id":"01M3YFNWZ3DR2G6KP9VMS6KE8G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:20.852Z","job_size_bytes":23,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-8bdc4686617c1caf6c1891a6: start","job_status":"start","queue_duration_s":0.288882,"scheduling_latency_s":0.28891}2379gitlab # [ 543.347878] gitaly[930]: time="2026-10-02T14:18:20.964Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFNWZ3DR2G6KP9VMS6KE8G grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:18:20.962" grpc.time_ms=1.201 pid=930 span.kind=server system=grpc2380gitlab # [ 543.358600] gitaly[930]: time="2026-10-02T14:18:20.991Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFNWZ3DR2G6KP9VMS6KE8G grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:18:20.990" grpc.time_ms=0.61 pid=930 span.kind=server system=grpc2381gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/727998be4a72c681de6dc36cedfc3d5e9518c280/statuses, in 0.16 seconds)2382(finished: subtest: eval and build post commit statuses, in 10.91 seconds)2383(finished: run the VM test script, in 618.45 seconds)2384gitlab # [ 543.369144] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:21.198Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"8bdc4686617c1caf6c1891a6","created_at":"2026-10-02T14:18:20.847Z","trace_propagation_headers":{"sentry-trace":"f160687c78284162b43077a6c064ed17-0c7b56dfa619473b","baggage":"sentry-trace_id=f160687c78284162b43077a6c064ed17,sentry-release=Unknown"},"correlation_id":"01M3YFNWZ3DR2G6KP9VMS6KE8G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-10-02T14:18:20.852Z","job_size_bytes":23,"pid":3989,"sidekiq_tid":"m961","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-8bdc4686617c1caf6c1891a6: done: 0.056772 sec","job_status":"done","queue_duration_s":0.288882,"scheduling_latency_s":0.28891,"redis_calls":7,"redis_duration_s":0.001625,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00113,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000495,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039199,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.056772,"completed_at":"2026-10-02T14:18:21.198Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013244,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2385test script finished in 618.49s2386cleanup2387kill QemuMachine (pid 46)2388gitlab # [ 543.418548] sidekiq-cluster[3989]: {"severity":"INFO","time":"2026-10-02T14:18:21.200Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"62e5fda4db374bb33cc1b10a","created_at":"2026-10-02T14:18:21.006Z","trace_propagation_headers":{"sentry-trace":"f160687c78284162b43077a6c064ed17-0c7b56dfa619473b","baggage":"sentry-trace_id=f160687c78284162b43077a6c064ed17,sentry-release=Unknown"},"correlation_id":"01qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/41m77i1296n33p6liin8ynr6wh3h6b7m-python3-3.14.6/bin/python3.14)2389(finished: cleanup, in 1.40 seconds)2390additionally exposed symbols:2391 gitlab,2392 vlan1,2393 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh2394[]2395[{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-10-02T14:16:15.654Z', 'push_events': True, 'tag_push_events': True, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]2396[]2397[]2398[]2399[]2400[]2401[]2402[]2403[]2404[]2405[{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '727998be4a72c681de6dc36cedfc3d5e9518c280', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'evaluating', 'eval_warnings': None, 'error': None, 'created_at': '2026-10-02T14:18:09.524322Z', 'started_at': '2026-10-02T14:18:09.981822Z', 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': None}]2406[{'id': 5, 'sha': '727998be4a72c681de6dc36cedfc3d5e9518c280', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-02T14:18:09.934Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2407[{'id': 5, 'sha': '727998be4a72c681de6dc36cedfc3d5e9518c280', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-02T14:18:09.934Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2408[{'id': 5, 'sha': '727998be4a72c681de6dc36cedfc3d5e9518c280', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-02T14:18:09.934Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2409[{'id': 5, 'sha': '727998be4a72c681de6dc36cedfc3d5e9518c280', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-02T14:18:09.934Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2410[{'id': 5, 'sha': '727998be4a72c681de6dc36cedfc3d5e9518c280', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-02T14:18:09.934Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2411[{'id': 5, 'sha': '727998be4a72c681de6dc36cedfc3d5e9518c280', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-02T14:18:09.934Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2412[{'id': 5, 'sha': '727998be4a72c681de6dc36cedfc3d5e9518c280', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-02T14:18:09.934Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2413[{'id': 5, 'sha': '727998be4a72c681de6dc36cedfc3d5e9518c280', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-02T14:18:09.934Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2414[{'id': 5, 'sha': '727998be4a72c681de6dc36cedfc3d5e9518c280', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 9s (1 warning)', 'created_at': '2026-10-02T14:18:09.934Z', 'started_at': None, 'finished_at': '2026-10-02T14:18:19.721Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': '727998be4a72c681de6dc36cedfc3d5e9518c280', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-10-02T14:18:19.947Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2415[{'id': 5, 'sha': '727998be4a72c681de6dc36cedfc3d5e9518c280', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 9s (1 warning)', 'created_at': '2026-10-02T14:18:09.934Z', 'started_at': None, 'finished_at': '2026-10-02T14:18:19.721Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': '727998be4a72c681de6dc36cedfc3d5e9518c280', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '0 attributes built', 'created_at': '2026-10-02T14:18:19.947Z', 'started_at': None, 'finished_at': '2026-10-02T14:18:20.224Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]